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ABSTRACT 


The  purpose  of  this  thesis  is  to  analyze  protests 
of  contract  awards  brought  before  the  Comptroller  General, 
General  Accounting  Office  from  January  1998  through 
December  1999  as  a  means  to  identify  areas  of  possible 
improvement  among  Federal  contracting  agencies.  Specific 
emphasis  is  on  the  underlying  causes  of  protest 
sustainment.  This  thesis  distills  eight  sustaining 
elements  from  the  GAO  findings.  Finally,  this  thesis 
offers  recommendations  to  Federal  contracting  agencies  in 
an  effort  to  help  mitigate  the  risk  of  a  sustainable 
contract  award  protest. 
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I .  INTRODUCTION 


A.  DISCUSSION 

Since  the  release  of  the  Federal  Acquisition 
Regulation  (FAR)  Part  15  rewrite,  Federal  contracting 
officers  have  been  encouraged  to  obtain  the  best  value  to 
the  Government  by  conducting  trade-offs  when  making  source 
selections.  If,  in  the  contracting  officer's  opinion,  it 
is  in  the  best  interests  of  the  Government  to  award  a 
contract  to  other  than  the  lowest-priced,  technically 
acceptable  offeror  or  to  the  highest  technically  rated 
offeror,  then  he  or  she  may  make  trade-offs.  One 
stipulation  for  making  trade-offs  is  that  they  be  properly 
documented  and  supported  by  a  reasoned  explanation. 

Even  assuming  that  a  contracting  officer  follows  this 
procedure,  there  is  nothing  to  stop  an  unsuccessful  offeror 
from  filing  a  protest  of  the  award  with  the  General 
Accounting  Office  (GAO).  Even  though  GAO's  decisions  are 
not  binding  on  Federal  agencies,  they  are  highly 
influential  and  signal  Congress's  intent  to  ensure  full  and 
open  competition  in  Government  procurement.  Protests 
arising  from  contract  awards  have  serious  potential  for 
both  adding  costs  to  an  acquisition  and  delaying  delivery 
to  the  end  user.  An  analysis  of  GAO's  decisions  should 
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reveal  ways  to  mitigate  the  risk  of  sustainable  contract 
award  protests. 

B.  OBJECTIVE 

This  study  will  benefit  both  current  and  prospective 
Federal  Government  contracting  officers  to  execute  their 
duties.  For  the  price  of  a  postage  stamp,  disgruntled 
would-be  contractors  can  bring  the  acquisition  process  to  a 
grinding  halt.  This  can  result  in  increased  costs  to  the 
Government  and  significant  delays  in  delivering  items  to 
the  end  user. 

Contracting  officers  will  be  better  prepared  to 
mitigate  the  risk  of  sustainable  contract  award  protests  if 
they  are  aware  of:  1)  the  historically  common  reasons  cited 
in  contract  protests;  and  2)  which  categories  of  protests 
of  awards  using  the  best  value  trade-off  process  have  high 
sustainment  rates. 

C.  RESEARCH  QUESTIONS 

1 .  Primary  Research  Question 

To  what  extent  will  an  analysis  of  General  Accounting 
Office  decisions  of  protests  of  contracts  awarded  using  the 
best  value  trade-off  process  provide  insight  into  the 
nature  of  Government  contract  protests  and  suggest  ways  to 
reduce  such  protests  in  the  future? 
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2. 


Secondary  Research  Questions 


•  What  will  an  analysis  of  Government  contract 
protests  from  January  1998  through  December  1999 
suggest  about  the  nature  of  Government  contract 
protests? 

•  Do  sustained  protests  of  contracts  awarded  using  the 
best  value  trade-off  process  fall  into  distinct 
categories? 

•  What  will  an  analysis  of  the  research  results 
suggest  about  weaknesses  in  Government  contracting 
norms  and  execution  practices? 

•  How  can  the  results  of  this  analysis  be  used  to 
reduce  the  risk  of  protest? 

•  How  can  the  results  of  this  analysis  be  used  to 
reduce  the  risk  of  a  sustainable  protest? 

D .  SCOPE 

This  thesis  will  include:  1)  a  review  of  all  decisions 
rendered  during  calendar  years  1998  and  1999  by  the 
Comptroller  General,  General  Accounting  Office,  in  response 
to  protests  of  contract  award;  and  2)  an  in-depth  analysis 
of  protests  of  contract  awards  using  the  best  value  trade¬ 
off  source  selection  process.  The  thesis  will  conclude 
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with  recommendations  to  contracting  officers  for  mitigating 
the  risk  of  sustainable  protests  of  contract  award. 

E.  METHODOLOGY 

This  study  was  conducted  using  the  following  methods: 

•  Research  using  LEXIS/NEXIS  to  identify  and  acquire 
applicable  cases. 

•  Review  and  analysis  of  each  protest  brought  before 
the  Comptroller  General  and  decided  during  calendar 
years  1998  and  1999.  Estimated  total:  558  cases. 

•  Thorough  examination  of  all  protests  of  contracts 
awarded  using  the  best  value  trade-off  source 
selection  process. 

•  Categorization  and  analysis  of  sustained  protests  in 
order  to  generate  recommendations  and  conclusions 
regarding  sustained  protests. 

F .  DEFINITIONS 

The  following  definitions  will  allow  the  reader  to 
better  understand  the  analysis  in  the  following  chapters. 

In  addition,  Appendix  A  provides  a  listing  of  selected 
acronyms  used  throughout  this  thesis. 

1 .  Acquisition 

"Acquisition"  means  the  acquiring  by  contract  with 
appropriated  funds  of  supplies  or  services  (including 
construction)  by  and  for  the  use  of  the  Federal  Government 
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through  purchase  or  lease,  whether  the  supplies  or  services 
are  already  in  existence  or  must  be  created,  developed, 
demonstrated,  and  evaluated.  Acquisition  begins  when 
agency  needs  are  established  and  includes  the  following: 
description  of  requirements  to  satisfy  agency  needs; 
solicitation  and  selection  of  sources;  award  of  contracts; 
contract  financing;  contract  performance;  contract 
administration;  and  those  technical  and  management 
functions  directly  related  to  the  process  of  fulfilling 
agency  needs  by  contract.  [Ref.  1:  p.P-3] 

2 .  Best  Value 

"Best  Value"  means  the  expected  outcome  of  an 
acquisition  that,  in  the  Government's  estimation,  provides 
the  greatest  overall  benefit  in  response  to  the  requirement 
for  supplies  or  services.  To  obtain  the  greatest  overall 
benefit  in  response  to  the  requirement,  one  must  consider 
the  impact  of  every  decision  and  action  on  price,  quality, 
timeliness,  risks,  competition,  socio-economic 
requirements,  and  compliance  with  ethical  standards. 

[Ref  1:  p.P-3] 

3 .  Contract 

A  contract  is  a  mutually  binding  legal  relationship 
obligating  the  seller  to  furnish  the  supplies  or  services 
(including  construction)  and  the  buyer  to  pay  for  them.  It 
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includes  all  types  of  commitments  that  obligate  the 
Government  to  an  expenditure  of  appropriated  funds;  these 
commitments  are,  except  as  otherwise  authorized,  in 
writing.  In  addition  to  bilateral  instruments,  contracts 
include  (but  are  not  limited  to)  awards  and  notices  of 
awards;  job  orders  or  task  letters  issued  under  basic 
ordering  agreements;  letter  contracts;  orders,  such  as 
purchase  orders,  under  which  the  contract  becomes  effective 
by  written  acceptance  or  performance;  and  bilateral 
contract  modifications.  Contracts  do  not  include  grants 
and  cooperative  agreements  covered  by  31  U.S.C.  §6301. 

[Ref.  2:2.101] 

4.  Interested  Party 

An  interested  party  is  an  actual  or  prospective  bidder 
or  offeror  with  a  direct  economic  interest  in  the 
procurement.  In  challenges  of  the  evaluation  of  proposals 
and  the  award  of  contracts,  this  generally  means  an  offeror 
that  would  be  in  line  for  award  if  the  protest  were 
sustained.  [Ref.  3:  p.9] 

5 .  Offer 

"Offer"  means  a  response  to  a  solicitation  that,  if 
accepted,  would  bind  the  offeror  to  perform  the  resultant 
contract.  Responses  to  invitations  for  bids  (sealed 
bidding)  are  offers  called  "bids"  or  "sealed  bids"; 
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responses  to  requests  for  proposals  (negotiation)  are 
offers  called  "proposals";  responses  to  requests  for 
quotations  (negotiation)  are  not  offers  and  are  called 
"quotes."  [Ref.  2:  2.1] 

6 .  Protest 

Protest  means  a  written  objection  by  an  interested 
party  to  any  of  the  following: 

•  A  solicitation  or  other  request  by  an  agency  for 
offers  for  a  contract  for  the  procurement  of 
property  or  services. 

•  The  cancellation  of  the  solicitation  or  other 
request . 

•  An  award  or  proposed  award  of  the  contract. 

•  Termination  or  cancellation  of  an  award  of  the 
contract  if  the  written  objection  contains  an 
allegation  that  the  termination  or  cancellation  is 
based  in  whole  or  in  part  on  improprieties 
concerning  the  award  of  the  contract. 

[Ref.  2:  33.101] 

7.  Source  Selection  Authority  (SSA) 

The  SSA  is  the  person  responsible  for  making  the 
source  selection  decision.  While  the  SSA  may  use  reports 
and  analyses  prepared  by  others,  the  source  selection 
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decision  must  represent  the  SSA's  independent  judgment. 
[Ref.  2:15.308] 

8.  Source  Selection  Evaluation  Board  (SSEB) 

The  team  that  actually  reviews  the  contractor's 
proposals  may  be  known  as  the  "Source  Selection  Evaluation 
Board  (SSEB),"  "Source  Evaluation  Board  (SEB),"  "Source 
Evaluation  Team  (SET),"  or  another  similar  name.  These 
experts  may  be  further  divided  into  sub-teams  to  evaluate 
different  aspects  of  each  contractor's  proposal  (e.g., 
cost/price,  technical,  and  past  performance).  [Ref.  4:  8.2] 

9 .  Trade-off  Process 

In  negotiated  procurements,  the  most  common  approach 
to  achieve  a  best-value  selection  is  a  trade-off  process  in 
which  dollars  are  traded  for  incremental  value,  and  cost  is 
only  one  factor,  not  the  controlling  factor.  A  trade-off 
process  is  appropriate  when  it  is  in  the  Government's  best 
interest  to  consider  award  to  other  than  the  lowest-priced 
offeror  or  other  than  the  highest  technically  rated 
offeror.  [Ref.  2:  15.101-1] 

G.  ORGANIZATION 

Chapter  I  of  this  thesis  presented  the  research 
questions  used  to  gain  insight  into  the  nature  of  protests 
of  contracts  awarded  using  the  best  value  trade-off 
process.  Additionally,  this  chapter  discussed  the  scope  of 
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the  thesis,  the  researcher's  methodology,  and  the  relevant 
terminology  used  throughout  this  study. 

Chapter  II  is  a  review  of  the  trade-off  process  used 
to  obtain  the  best  value  for  Federal  Government  contracts, 
including  the  origin  of  trade-offs,  the  development  of  the 
trade-off  process,  and  the  integrity  of  the  process.  The 
chapter  also  provides  a  brief  overview  of  the  protest 
process . 

Chapter  III  is  a  quantitative  analysis  of  sustained 
protests  of  contracts  awarded  using  the  best  value  trade¬ 
off  process  for  calendar  years  1998  and  1999,  including  a 
description  of  the  methodology  used  to  obtain  the  data. 

Chapter  IV  will  present  a  qualitative  analysis  of 
sustained  protests  of  contracts  awarded  using  the  best 
value  trade-off  process  for  calendar  years  1998  and  1999. 
This  chapter  also  categorizes  the  recurring  reasons  for 
sustained  protests  and  identifies  common  weaknesses  that 
contributed  to  these  protests  being  sustained. 

Chapter  V  presents  the  conclusions  and  recommendations 
generated  from  this  research,  as  well  as  suggested  areas 
for  further  research. 
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II .  BACKGROUND 


A.  INTRODUCTION 

This  chapter  will  present  an  overview  of  the  trade-off 
process  used  to  obtain  the  best  value  for  Federal 
Government  contracts.  The  purpose  of  this  chapter  is  to 
provide  the  reader  with  a  basic  understanding  of  the  trade¬ 
off  process.  For  a  more  detailed  discussion,  see  Appendix 

B,  which  contains  excerpts  from  "AMC  pamphlet  715-3: 
Contracting  for  Best  Value;  A  Best  Practices  Guide  to 
Source  Selection."  Specifically,  this  chapter  will  stress 
the  importance  of  maintaining  integrity  in  the  trade-off 
process.  Finally,  the  chapter  will  provide  a  brief 
overview  of  the  protest  process. 

1 .  Review  of  the  Trade-off  Process  in  Contracting 
Acquisition  professionals,  over  use  the  term  "Best- 
value"  as  it  is  commonly  encountered.  According  to  one 
source,  "Best-value"  means  buying  on  other  than  a  price- 
only  basis,  the  usual  Government  practices ." [Ref .  5:  p.3-1] 
Every  acquisition  brokered  on  behalf  of  the  Federal 
Government  must  be  awarded  "to  the  responsible  source  whose 
proposal  is  most  advantageous  to  the  United  States, 
considering  only  cost  or  price  and  the  other  factors 
included  in  the  solicitation." 
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[Ref.  3:  U.S.C.  §2305 (b)  ( 4 )  (c) ] .  By  definition,  a  contract 
award  must  be  determined  to  be  in  the  Government's  best 
interests.  In  other  words,  a  best-value  award  is  one  that 
is  made  to  the  offeror  proposing  the  lowest  price  or  the 
highest  technically  rated  offer,  or  some  combination  of  the 
two  as  long  as  the  award  is  determined  to  be  the  most 
advantageous  to  the  United  States.  For  this  reason,  the 
researcher  has  chosen  to  use  the  term  "trade-off  process" 
to  identify  contracts  awarded  using  the  trade-off 
decisional  rule. 

A  trade-off  process  is  appropriate  when  it  is 
essential  to  evaluate  and  compare  factors  other  than  cost 
or  price  in  order  to  select  the  proposal  that  is  in  the 
Government's  best  interest.  Trade-offs  sometimes  mean  that 
awards  are  not  made  to  those  who  offer  the  lowest  price  or 
the  highest  technically  rated  offeror.  [Ref.  2:  15.101(a)] 

The  following  are  instances  in  which  the  trade-off 
process  may  be  particularly  appropriate: 

•  Government  requirements  are  complex  or  difficult  to 
define. 

•  Services  are  not  clearly  defined,  or  highly  skilled 
personnel  are  required. 
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•  The  Government  is  willing  to  pay  extra  for 
capability,  skills,  or  reduced  risk  if  the  added  benefits 
are  worth  the  price  premium.  [App.  B:p. 120-121] 

A  best-value  award  using  the  trade-off  process  then 
falls  along  the  continuum  that  includes:  1)  lowest  priced, 
technically  acceptable  (LPTA)  and  2)  highest  technically 
rated  offer.  A  frequently  cited  range  of  alternatives — the 
best-value  continuum — is  often  used  to  describe  this  trade¬ 
off  of  alternatives  along  the  spectrum  of  options  noted 
above.  [Ref.  2:  15.101] 

For  the  purposes  of  this  thesis,  all  contract  awards 
discussed  will  be  those  evaluated  solely  using  the  trade¬ 
off  process. 

2 .  Integrity  of  the  Trade-off  Process 

One  of  the  fundamental  concerns  of  Government 
contracting  officers  is  maintaining  the  public's  trust— the 
trust  of  prospective  contractors  as  well  as  of  the  general 
public.  Maintaining  the  integrity  of  the  procurement 
process  is  not  only  an  ethical  issue,  but  it  is  also  a 
legal  one  since  there  are  both  civil  and  criminal  penalties 
for  violations.  [App.  B:  p.116] 

One  way  contracting  officers  in  negotiated 
procurements  maintain  this  trust  is  to  award  contracts 
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solely  in  accordance  with  the  stated  evaluation  criteria 
specified  in  each  solicitation.  [Ref:  2:1.102] 

Specifically,  all  factors  and  significant  sub-factors 
that  will  affect  contract  award  must  be  stated  clearly  in 
the  solicitation,  along  with  their  relative  importance. 
Negotiated  procurements  can  be  complex,  including  first- 
level  evaluators,  a  source  selection  evaluation  board 
(SSEB) ,  a  source  selection  advisory  council  (SSAC)  and, 
ultimately,  the  source  selection  authority  (SSA) .  The 
illustration  on  page  six  of  Appendix  B  provides  an  example 
of  a  complex  source  selection  team. 

Alternatively,  negotiated  procurements  can  be  less 
complex  and  include  only  first-level  evaluators  and  the 
SSA.  The  diagram  on  page  seven  of  Appendix  B  illustrates  a 
typical  non-complex  source  selection  team. 

[Ref.  2:15. 304(d)] 

By  evaluating  each  proposal  strictly  against  the 
factors  and  sub-factors  stated  in  the  solicitation,  and  not 
by  comparison  to  the  proposals  of  other  offerors,  the 
first-level  evaluators  do  their  part  to  maintain  the 
integrity  of  the  source  selection  process.  Depending  on 
the  complexity  of  the  procurement,  the  first-level 
evaluators  provide  the  results  of  their  evaluation  either 
to  the  SSAC  or  directly  to  the  SSA.  For  procurements  that 
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require  a  complex  source  selection  team,  the  evaluators 
forward  their  evaluation  to  the  SSAC,  which  will  compare 
the  various  offers  and  make  an  award  recommendation  to  the 
SSA.  In  less-complex  procurements,  the  SSA  receives  the 
evaluations  directly  from  the  first-level  evaluators  and 
performs  the  tasks  of  both  comparing  offers  and  selecting 
the  offer (s)  for  award.  By  adhering  to  this  policy,  the 
source  selection  team  helps  ensure  that  the  public's  trust 
is  maintained. 

The  SSA  then — as  the  final  decision  authority — must 
make  an  award  decision  based  on  a  comparative  assessment  of 
proposals  against  all  source  selection  criteria.  The  SSA 
must  consider  all  available  information,  including  reports 
and  analyses  prepared  by  others,  in  making  this  decision. 
This  award  decision  must  represent  the  SSA' s  independent 
judgment  and  must  be  adequately  documented.  The 
documentation  should  include  the  rationale  for  any  business 
judgments  and  trade-offs  made  by  the  SSA,  as  well  as  the 
benefits  associated  with  any  price  premiums  accepted. 

[Ref.  2:15.308] 

B .  PROTEST  PROCESS 

Any  interested  party  in  the  acquisition  process, 
either  because  of  frustrated  feelings  or  suspicions  of 
unfair  treatment,  may  file  a  formal  protest.  Such  protests 
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may  cost  the  Government  a  lot  of  time  and  administrative 
effort,  as  well  as  the  involvement  of  the  entire 
acquisition  team.  Additionally,  with  limited  exceptions,  a 
protest  will  suspend  performance  of  the  contract  pending  a 
decision  by  the  agency  or  the  Comptroller  General.  It  is, 
therefore,  in  everyone's  best  interests  to  avoid  any 
actions — perceived  or  actual — that  could  prompt  a 
prospective  contractor  to  file  a  formal  protest. 

Contractors  have  several  avenues  for  attempting  to 
resolve  contract  award  controversies.  This  thesis  will 
consider  only  those  protests  filed  with  the  Comptroller 
General  for  calendar  years  1998  and  1999.  The  following 
paragraphs  will  outline:  1)  how  a  formal  protest  is  filed; 
2)  an  agency's  action  upon  receiving  a  protest;  3)  avenues 
open  to  the  protestor  upon  receiving  the  agency's  response; 
and  4)  the  protest  hearing  process  and  the  General 
Accounting  Office's  (GAO)  decision.  Table  1  below  presents 
a  snapshot  of  the  intricacies  involved  when  a  contract 
award  is  contested. 
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TABLE  1. 

CONTRACT  AWARD  CONTROVERSIES 


|  CONTRACT  AWARD  CONTROVERSIES  | 

Forum 

Procuring 

Agency 

Court  of 
Federal  Claims 
(COFC) 

District  Courts 

GAO 

Who  May 

Protest 

Interested 

parties 

Persons  with 
implied 
contract  for 
good  faith 
consideration 
of  offer  and 
interested 
parties 

Persons 
adversely 
affected  or 
aggrieved  by 
agency  action 
and  interested 
parties 

Interested 

parties 

Jurisdiction 

FAR  33.103 

28  U.S.C. 

§1491 (a) (1)  and 
(b)  (2) 

5  U.S.C.  §702, 

28  U.S.C. 

§1346(2)  and  28 
U.S.C. 

§1491 (b) (2) 

31  U.S.C. 

§3551,  with 
deference  to 
actions  in 
other  forums 

Time  for 

Protest 

Pre-  and  post 
award 

Pre-  and  post 
award 

Pre-  and  post 
award 

Pre-  and  post 
award 

Time  of 
Decision 

No  designated 
time 

No  designated 
time 

No  designated 
time 

100  calendar 
days  (65 
calendar  days  - 
express  option) 

Procedures 

Various  agency 
procedures; 
Informal  in 
nature 

Evidentiary 

trial, 

including 

discovery 

Evidentiary 
trial,  including 
discovery 

Submission  of 
written  reports 
and  optional 
hearing 

Burden  of 

Proof 

Probably  same 
as  GAO 

;  Clear  and 
convincing 
evidence  of 
violation 

Preponderance  of 
;  evidence 

Presumption  of 
correctness  of 
agency  action 

Standards  of 
Review 

Violation  of 
statute  or 
regulation 

No  rational 
basis  or  clear 
violation  of 
statute  or 
regulation 

No  rational 
basis  or  clear 
violation  of 
statute  or 
regulation 

Violation  of 
statute  or 
regulation 

Interim 

Remedies 

:  Agency  must 
delay  award  or 
suspend 
:  performance 
unless  CICA 
override 

Discretionary 

temporary 

restraining 

order  or 

preliminary 

injunction 

Discretionary 

temporary 

restraining 

order  or 

preliminary 

injunction 

Agency  must 
delay  award  or 
suspend 
performance 
unless  CICA 
override 

Remedies 

Any 

appropriate 

actions 

Injunction  or 
declaratory 
judgment  if 
consistent  with 
national 

i  defense;  bid  or 
proposal  costs 

Injunction  or 
declaratory 
judgment,  unless 
inconsistent 
with  overriding 
public  interest, 

I  bid  or  proposal 
costs  (limited 
to  $10,000  under 
§1346(2) ) 

Recommend  that 
agency 
recompete, 
terminate,  or 
award;  contract 
costs; 
attorney' s 
fees;  bid  or 
proposal  costs 

Appeals 

If  within 
jurisdictional 
time  period, 
begin  action 
again  in 
either  GAO, 

COFC,  or 
district  court 

Federal  Circuit 

Applicable  court 

Indirectly 
through  suit  in 
the  COFC  or 
district  court 

[Source:  Ref.  [6:p796] 


17 


1. 


Protest  Filed 


The  first  recourse  available  to  a  prospective 
contractor  who  feels  unfairly  treated  is  to  file  a  formal 
written  protest  to  the  agency  that  issued  the  solicitation. 
This  formal  protest  must  contain,  among  other  things,  a 
detailed  statement  of  the  legal  and  factual  grounds  for  the 
protest,  including  a  description  of  resulting  prejudice  to 
the  protestor.  [Ref. 2  : 33 -  103(d)  (2)] 

If  the  protest  is  based  on  alleged  improprieties  in  a 
solicitation,  and  a  protest  is  not  filed  prior  to  the 
closing  date  for  receipt  of  proposals,  such  a  protest  will 
be  considered  "untimely  filed"  and  will  normally  be  denied. 
In  all  other  cases,  protests  shall  be  filed  no  later  than 
ten  days  after  the  basis  of  protest  is  known,  or  should 
have  been  known,  whichever  is  earlier.  Becoming  aware  of 
the  basis  for  a  protest  normally  occurs  during  the 
debriefing  process.  See  Appendix  B,  pages  179-202  for  a 
thorough  understanding  of  the  debriefing  process.  [Ref. 2: 
33.103 (e) ] 

If  the  agency  denies  the  protest,  then  the  protestor 
has  the  option  of  pursuing  the  protest  with  the  GAO.  If 
the  protestor  decides  take  the  protest  to  GAO,  then  he  or 
she  must  file  it  not  later  than  ten  days  after  learning  of 
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2 .  Agency  Response 

Upon  receiving  a  protest  before  award,  unless  the 
acquisition  fulfills  an  urgent  and  compelling  need  or  is 
otherwise  in  the  Government's  best  interest,  the 
contracting  officer  will  withhold  award  pending  resolution 
of  the  protest. 

If  the  protest  is  received  within  ten  days  after 
contract  award,  or  within  five  days  after  a  debriefing  date 
offered  to  the  protestor,  the  contracting  officer  shall 
suspend  performance  of  the  contract  pending  resolution  of 
the  protest.  Here,  too,  the  same  exceptions  of  "urgent  and 
compelling  need"  or  "best  interest  of  the  Government" 
apply.  [Ref:  2  :  33 . 103 (f ) ] . 

The  agency  then  must  make  every  effort  to  resolve  the 
agency-level  protest  within  35  days  of  receipt  of  the 
protest . 

3 .  Protestor  Rebuttal 

A  protestor  who  remains  dissatisfied  after  resolution 
of  his  or  her  protest  at  the  agency  level  may  then  file 
protest  at  GAO.  From  this  date,  the  agency  has  30  days  to 
submit  to  GAO  (with  copies  to  the  protestor  and  any  other 
interested  parties)  a  complete  written  report  responding  to 
the  protest,  including  all  relevant  documents  and  an 
explanation  of  the  agency's  position.  The  protestor  may 
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then  file  written  comments  on  the  agency  report.  Should 
the  protestor  fail  to  file  comments,  or,  alternatively, 
request  in  writing  that  the  case  be  decided  on  the  existing 
written  record,  the  case  will  be  dismissed.  [Ref . 3 :p. 28-31] 

4 .  GAO  Hearings  and  Decision 

GAO  may,  either  at  the  request  of  a  party  or  on  its 
own  initiative,  conduct  hearings  in  person  or  by  telephone. 
Due  to  the  increased  costs  and  burdens  they  create,  such 
hearings  are  held  only  when  necessary.  If  GAO  holds  a 
hearing,  all  parties  will  be  permitted  to  file  written 
comments  on  the  hearing. 

Once  GAO  has  all  the  information  necessary  to  decide 
the  protest,  it  will  issue  a  written  decision  through  the 
Comptroller  General  within  100  days  after  the  protest  is 
filed.  GAO's  decision  will  be  to  dismiss,  deny  or  sustain 
the  protest.  The  general  criterion  for  GAO  to  sustain  a 
protest  is  that  the  contracting  agency  violated  procurement 
statutes  or  regulations.  GAO  will  also  recommend 
appropriate  corrective  action  for  those  cases  it  sustains. 
Finally,  GAO  will  recommend  that  the  protestor  be 
reimbursed  for  the  costs  of  filing  and  pursuing  the 
protest.  [Ref.  3:  p37-38] 

GAO's  decisions  are  not  binding,  but  are  merely 
recommendations  to  which  the  contracting  agency  is  not 
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required  to  adhere.  The  Comptroller  General  does,  however, 
require  agencies  to  respond  in  writing  whenever  they  choose 
not  to  follow  GAO's  recommendations.  The  Comptroller 
General,  in  turn,  reports  to  Congress  annually  on  each 
instance  where  its  recommendations  were  not  fully 
implemented.  In  this  way,  the  GAO  exercises  significant 
influence  over  the  actions  of  the  contracting  agencies. 

[Ref.  3:  p.38] 

C .  CONCLUSION 

This  chapter  first  provided  a  brief  overview  of  the 
trade-off  process.  For  a  more  comprehensive  understanding 
of  contracts  awarded  using  the  trade-off  process,  see 
Appendix  B,  which  contains  excerpts  from  the  Army  Material 
Command's  pamphlet  715-3  entitled  "Contracting  For  Best 
Value,  A  Best  Practices  Guide  To  Source  Selection."  This 
pamphlet  is  an  excellent  reference  guide  to  the  trade-off 
source  selection  process,  from  planning  the  procurement 
through  debriefing  offerors. 

Next,  this  chapter  presented  highlights  of  the  protest 
process  as  it  pertains  to  protests  filed  with  the  General 
Accounting  Office.  Competitive  Negotiation ,  The  Source 
Selection  Process,  by  Cibinic,  Nash  and  O'Brien,  and  Bid 
Protests  At  GAO:  A  Descriptive  Guide,  by  the  Office  of 
General  Counsel,  GAO  both  are  outstanding  sources  for  a 
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comprehensive  discussion  of  contract  award  controversies 
and  the  bid  protest  process  at  GAO,  respectively. 
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III.  QUANTITATIVE  ANALYSIS  OF  PROTESTS  FOR  CALENDAR 
YEARS  1998  AND  1999 

A.  INTRODUCTION 

This  chapter  provides  an  overview  of  the  quantitative 
results  of  the  data  gathered  from  cases  decided  by  the 
Comptroller  General  for  calendar  years  1998  and  1999.  The 
chapter  first  presents  the  methodology  used  for  collecting 
data.  Then,  it  explains  the  rationale  for  narrowing  the 
population  to  protests  of  contracts  awarded  using  the 
trade-off  process.  Finally,  the  chapter  provides 
statistics  to  give  the  reader  an  understanding  of  the 
breakdown  of  the  sustained  protests  studied. 

B.  DATA  COLLECTION  METHODOLOGY 

The  global  sample  population  for  this  study  consisted 
of  558  protests  decided  by  the  Comptroller  General  between 
01  January  1998  and  31  December  1999. 

These  cases  were  drawn  from  the  LEXIS-NEXIS  system  by 
using  the  following  queries  within  the  Comptroller  General 
sub-directory  of  the  General  Federal  directory: 

•  "Protest"  AND  "date>12/31/97"  AND  "date<7/l/98" 

•  "Protest"  AND  "date>6/30/98"  AND  "date<l/l/99" 

•  "Protest"  AND  "date>12/31/98"  AND  "date<7/l/99" 

•  "Protest"  AND  "date>6/30/99"  AND  "date<l/l/00" 
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The  researcher  used  four  queries  to  extract  the 
necessary  data  because  the  LEXIS-NEXIS  system  allows  no 
more  than  200  cases  to  be  printed  at  one  time.  The 
resulting  queries  distributed  the  global  population 
relatively  evenly. 

From  this  global  population,  the  researcher  analyzed 
each  case  to  determine  if  a  trade-off  process  was  used  in 
evaluating  solicitations  and  awarding  the  subject  contract. 
This  process  narrowed  the  population  to  325  cases.  Appendix 
C  provides  a  complete  listing  of  these  cases. 

Once  these  protests  were  identified,  various  data 
elements  were  captured  for  use  in  analyzing  the  cases 
further.  In  order  to  standardize  this  data  collection,  the 
researcher  created  a  data  collection  form  that  contained 
the  following  data  fields: 

•  GAO  case  number 

•  Case  name 

•  Decision  date 

•  Decision 

•  Award  Agency 

•  RFP  number 

•  Reason  for  protest 

•  Weaknesses  identified 
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Next,  the  researcher  selected  protests  that  the 
Comptroller  had  either  completely  or  partially  sustained, 
resulting  in  a  target  population  of  47  cases.  Finally,  the 
researcher  eliminated  five  cases  whose  basis  for  protest 
did  not  meet  his  definition  of  an  element  of  the  trade-off 
process  (e.g.,  competitive  range  determinations,  small 
business  set-aside  decisions,  etc.)  to  arrive  at  the  final 
target  population  of  42  cases.  Figure  1  below  provides  a 
graphic  representation  of  the  narrowing  of  the  global 
population  down  to  the  final  target  population. 
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FIGURE  1.  DETERMINING  STUDY  TARGET  POPULATION 
[Source:  Developed  by  researcher.] 


C.  METHODOLOGY  FOR  SELECTING  THE  PROTESTS  OF  CONTRACTS 

AWARDED  USING  THE  TRADE-OFF  PROCESS 

As  stated  above,  the  Comptroller  General  sustained  47 
protests  of  contracts  awarded  using  the  trade-off  process 
during  the  time  period  studied.  Of  these,  five  were 
protests  of  peripheral  issues  rather  than  of  the  trade-off 
process  itself  and,  therefore,  were  eliminated.  Table  2 
lists  the  five  cases  that  were  eliminated  from  the  target 
population,  as  well  as  the  element  protested. 


26 


TABLE  2.  PROTESTS  FAILING  TO  MEET  DEFINITION 


[Source : 

:  Developed  by  researcher.] 

Name 

Reason  for  protest 

1 

Aalco  Forwarding 

RFP  not  properly  set-aside  for 

small  business. 

2 

Hoechst  Marion 

Russel,  Inc. 

Requirements  stated  in  RFP  do  not 

realistically  state  the  Agency's 

requirements . 

3 

Technical  Support 

Services,  Inc. 

RFP  did  not  state  lowest  priced, 

technically  acceptable  as  award 

basis  yet  awarded  as  such. 

4 

Pemco  Aeroplex, 

Inc. 

Unduly  restrictive  of  competition 

due  to  combining  requirements. 

5 

Dual,  Inc. 

Awardee  made  misrepresentations  in 

offer  and  agency  relied  upon  these 

in  its  evaluation. 

D .  OVERALL  STATISTICS 

Of  the  558  total  protests  for  the  calendar  years  1998 
and  1999,  81  were  sustained.  The  sustainment  rate  for 
award  protests  in  general  is,  then,  14  percent  (14.34 
percent) .  This  compares  to  325  protests  of  contracts 
awarded  using  the  trade-off  process,  of  which  47  were 
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ultimately  sustained.  The  researcher  did  not  include  the 
remaining  233  protests  filed  because  they  were  not  protests 
of  contracts  awarded  using  the  best  value  trade-off  process 
as  their  source  selection  method.  Rather,  they  were  either 
awards  that  used  sealed  bidding  or  lowest 
priced/technically  acceptable  as  their  source  selection 
method,  or  protests  of  issues  not  concerning  the  trade-off 
process  in  source  selection  (e.g.,  requests  to  reconsider  a 
previous  decision,  requests  to  establish  the  amount  to  be 
reimbursed  resulting  from  a  previous  decision,  etc.).  Of 
the  47  remaining  in  the  target  population,  five  did  not 
involve  the  trade-off  process.  This  gives  a  cumulative 
sustainment  rate  for  trade-off  process  protests  of  13 
percent  (12.92  percent)  for  calendar  years  1998  and  1999. 

Considering  all  protests  of  contracts  awarded  for 
calendar  year  1998  independently,  328  were  decided  by  the 
Comptroller  General.  Of  these  328  protests,  48  were 
sustained,  giving  a  sustainment  rate  of  15  percent  (14.63 
percent)  for  all  protests. 

By  comparison,  there  were  196  protests  of  contracts 
awarded  using  the  trade-off  process.  Of  these,  168  were 
denied  or  dismissed,  five  were  sustained,  but  did  not 
involve  the  trade-off  process,  and  23  were  sustained 
protests  that  challenged  a  trade-off  process  element.  This 
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gives  a  sustainment  rate  of  12  percent  (12.04  percent)  for 
calendar  year  1998. 

Similarly,  there  were  a  total  of  230  protests  in 
calendar  year  1999.  Of  these,  33  were  decided  in  favor  of 
the  protestor  (sustained),  providing  a  sustainment  rate  of 

14  percent  (14.35  percent)  for  that  year. 

This  compares  to  a  total  of  129  protests  of  contracts 
awarded  using  the  trade-off  process.  Of  these,  19  were 
sustained  and  met  the  researcher's  definition  of  protests 
of  contracts  awarded  using  the  trade-off  process.  This, 
then,  yields  a  sustainment  rate  for  calendar  year  1999  of 

15  percent  (14.73  percent) .  Table  3  provides  a  summary  of 
the  sustainment  rates  for  specific  periods  and  protest 
classifications.  Figure  2  graphically  presents  the 
information  for  both  years  studied. 
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TABLE  3.  GENERAL  AND  SPECIFIC  SUSTAINMENT  RATES  FOR  PERIODS 

STUDIED 

[Source:  Developed  by  researcher.] 


Ca  tegory 

No. 

1998 

Of  Cases  and 

1999 

Percentages 

Combined 

Total  cases 

328 

230 

558 

Total  Sustained 

48/15% 

33/14% 

81/14% 

Trade-off 

process  cases 

196 

129 

325 

Sustained  trade¬ 
off  process 

protests 

23/12% 

19/15% 

42/13% 

30 


500" 


400' 


300 


200' 


100 


Total  Total  Trade-off  Sustained 

Cases  Sustained  Process  Trade-off 

(14%)  Cases  Protests 

(13%) 

Figure  2.  PROTEST  DATA  FOR  CY98/99 
[Source:  Developed  by  researcher.] 

By  further  analyzing  only  the  remaining  42  sustained 
protests,  the  researcher  derived  the  following  information. 
Analysis  of  the  type  of  Government  agency  that  awarded  the 
contracts  resulting  in  sustained  protests  provides  the 
following  data.  Exactly  half  of  the  sustained  protests  were 
for  contracts  awarded  by  an  Agency  within  the  Department  of 
Defense  (DoD) .  The  other  half  of  the  contracts  were  awarded 
by  various  other  Federal  agencies.  However,  considering 
the  number  of  protests  of  contracts  awarded  using  the 
trade-off  process,  the  DoD  had  a  lower  sustainment  rate 
than  the  other  Federal  agencies  as  a  whole  over  both  years — 
11  percent  (11.11  percent)  for  the  DoD  compared  to  15 


Number 
Of  Cases 
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percent  (15.44  percent)  for  all  other  Federal  agencies.  The 
remainder  of  the  statistical  analysis  will  treat  these  two 
groups — DoD  and  Other  Federal  Agencies  (OFA)  — as  distinct 
segments  of  the  target  population  for  comparison  purposes. 
Table  4  provides  a  tabular  breakdown  of  this  information. 


Table  4 .  Breakdown  of  protests  and  sustainment  rates  by  DoD 
component  compared  to  other  Federal  agencies. 

'  Source.  Developed  by  the  researcher. 


DoD 

Other  Fed. 
Total 


DoD 

Other  Fed. 
Total 


DoD 

Other  Fed. 
Total 


Number  of 
Protests 
108 
90 
198 


Number  of 
Protests 
81 
46 
127 


Number  of 
Protests 
189 
136 
325 


1998 
Number 

Sustained 

9 

14 

23 

1999 
Number 

Sustained 

12 

7 

19 

Cumulative 

Number 

Sustained 

21 

21 

42 


Sustainment 

Rate 

8.33% 

15.56% 

11.62% 


Sustainment 

Rate 

14.81% 

15.22% 

14.96% 


Sustainment 

Rate 

11.11% 

15.44% 

12.92% 
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E.  STATISTICS  BY  AGENCY 

Analysis  of  the  data  within  particular  agencies 
provides  the  following  results.  Within  the  Department  of 
Defense,  the  Navy  accounted  for  nine  sustained  protests  of 
trade-off  awards  for  calendar  years  1998  and  1999.  As  a 
percentage  of  all  sustained  protests  of  contracts  awarded 
by  the  DoD  using  the  trade-off  process,  the  Navy  awarded  43 
percent  (42.86  percent)  of  the  contracts  for  the  period 
studied.  Among  protests  of  Navy  contracts  awarded  using  the 
trade-off  process,  there  is  a  sustainment  rate  of  15 
percent  (14.52  percent)  for  calendar  years  1998  and  1999. 

Similarly,  of  the  21  sustained  protests  of  contracts 
awarded  using  the  trade-off  process  by  DoD  agenciesduring 
this  period,  the  Army  was  responsible  for  six,  or  29 
percent  (28.57  percent).  These  numbers  equate  to  a 
sustainment  rate  of  ten  percent  (9.84  percent)  for  protests 
of  Army  contracts  awarded  using  the  trade-off  process. 

Still,  within  the  Department  of  Defense,  two  other 
agencies  contributed  to  the  DoD  total  with  more  than  one 
sustained  protest.  The  Air  Force  and  the  Defense  Finance 
and  Accounting  Service  (DFAS)  each  accounted  for  two 
sustained  protests.  The  DFAS  protests  occurred  in  calendar 
year  1998  and  accounted  for  22  percent  (22.22  percent)  of 
the  DoD  total  of  nine  sustained  protests  that  year.  With  a 
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total  of  two  protests  of  DFAS  contracts  awarded  using  the 
trade-off  process  in  that,  this  produces  a  100  percent 
sustainment  rate.  In  calendar  year  1999,  the  study  reveals 
a  single  protest  of  a  DFAS  trade-off  award  that  was 
ultimately  denied  by  the  Comptroller  General.  Similarly, 
the  Air  Force  logged  two  sustained  protests  in  calendar 
year  1999,  which  reflected  17  percent  (16.67  percent)  of 
the  DoD  total  of  12.  Given  that  there  were  45  protests  of 
Air  Force  trade-off  contracts  awarded  for  the  two  years 
studied,  this  gives  a  sustainment  rate  of  four  percent 
(4.44  percent).  Table  5  provides  a  summary  of  these 
results . 
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Table  5.  Summary  of  protests  and  protest  sustainment 
rates  within  the  Department  of  Defense. 
Source.  Developed  by  the  researcher. 


1998 


Air  Force 
Army 
DFAS 
Navy 

Other  DoD 
DoD  Total 


Air  Force 
Army 
DFAS 
Navy 

Other  DoD 
DoD  Total 


Air  Force 
Army 
DFAS 
Navy 

Other  DoD 
DoD  Total 


Number  of 
Protests 
26 
37 
2 

32 

11 

108 


Number  of 
Protests 
19 
24 
1 

30 

7 

81 


Number  of 
Protests 
45 
61 
3 

62 

18 

189 


Number 

Sustained 

0 

2 

2 

4 

1 

9 

1999 

Number 

Sustained 

2 

4 
0 

5 
1 

12 


CUMULATIVE 

Number 

Sustained 

2 

6 

2 

9 

2 

21 


Sustainment 

Rate 

0.00% 

5.41% 

100.00% 

12.50% 

9.09% 

8.33% 


Sustainment 

Rate 

10.53% 

16.67% 

0.00% 

16.67% 

14.29% 

14.81% 


Sustainment 

Rate 

4.44% 

9.84% 

66.67% 

14.52% 

11.11% 

11.11% 


Among  Other  Federal  Agencies  (OFA) ,  only  four  of  the 
13  agencies  had  more  than  one  sustained  protest.  These 
were  the  Department  of  Health  and  Human  Services  (HHS) ,  the 
Department  for  Housing  and  Urban  Development  (HUD) ,  the 
Department  of  Agriculture  (DoA)  and  the  General  Services 
Administration  (GSA) . 

The  Departments  of  Health  and  Human  Services  and 
Housing  and  Urban  Development  topped  the  list  with  four 
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sustained  protests  each.  This  equates  to  19  percent  (19.05 
percent)  of  all  sustained  protests  within  the  OFA  sub¬ 
group.  The  difference  between  the  two  agencies,  in  terms 
of  sustained  protests,  lies  in  the  number  of  protests  filed 
against  them — ten  for  HHS  and  11  for  HUD.  This  yields 
sustainment  rates  of  40  percent  (40.00  percent)  and  36 
percent  (36.36  percent),  respectively.  The  next  Federal 
agency  that  experienced  more  than  a  single  sustained 
protest  was  the  Department  of  Agriculture  (DoA) ,  which  had 
three.  That  accounted  for  14  percent  (14.29  percent)  of 
all  OFA  sustained  protests  and  gave  DoA  a  sustainment  rate 
of  20  percent  (20.00  percent),  given  that  there  were  15 
protests  filed  against  their  awarded  contracts  for  the 
relevant  period.  The  only  other  Federal  agency  to  receive 
more  than  a  single  unfavorable  decision  from  the 
Comptroller  General  in  response  to  a  filed  protest  was  the 
General  Services  Administration  (GSA) .  GSA  had  two  of  the 
15  protests  brought  against  it  sustained,  or  13  percent 
(13.33  percent).  As  a  percentage  of  all  OFA  sustained 
protests,  GSA  accounted  for  ten  percent  (9.52  percent). 
Table  6  depicts  the  number  of  protests,  as  well  as  the 
number  of  those  sustained,  among  agencies  other  than  the 
DoD,  for  the  subject  calendar  years. 
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Table  6.  Summary  of  protests  and  protest  sustainment  rates 
among  other  Federal  agencies. 

Source:  Developed  by  the  Researcher 


DoA 

GSA 

HHS 

HUD 

Misc.  0FA 
0FA  Total 


DoA 

GSA 

HHS 

HUD 

Misc.  0FA 
0FA  Total 


DoA 

GSA 

HHS 

HUD 

Misc.  0FA 
0FA  Total 


Number  of 
Protests 
10 
9 
9 
7 

56 

90 


Number  of 
Protests 

5 

6 
1 
4 

30 
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Number  of 
Protests 
15 
15 
10 
11 
86 
136 


1998 
Number 

Sustained 

3 
0 

4 
2 
6 

14 

1999 
Number 

Sustained 

0 

2 

0 

2 

3 

7 


CUMULATIVE 

Number 

Sustained 

3 
2 

4 
4 
9 

21 


Sustainment 

Rate 

30.00% 

0.00% 

44.44% 

28.57% 

10.71% 

15.56% 


Sustainment 

Rate 

0.00% 

33.33% 

0.00% 

50.00% 

10.00% 

15.22% 


Sustainment 

Rate 

20.00% 

13.33% 

40.00% 

36.36% 

10.47% 

15.44% 


F.  CONCLUSION 

This  chapter  displays  the  quantitative  results  of  the 
researcher's  analysis.  The  overall  sustainment  rate  of 
protests  of  contracts  brought  before  the  Comptroller 
General  was  14  percent  for  calendar  years  1998  and  1999 
cumulatively.  By  comparison,  the  sustainment  rate  for 
protests  of  contracts  awarded  using  a  trade-off  process  for 
the  same  period  was  13  percent.  In  other  words,  whether  a 
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contract  was  awarded  using  the  LPTA  approach,  sealed  bids, 
or  the  trade-off  process,  there  was  virtually  no  difference 
in  the  rate  at  which  protestors  received  a  favorable 
decision. 
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IV.  QUALITATIVE  ANALYSIS  OF  PROTESTS  FOR  CALENDAR  YEARS 

1998  AND  1999 

A.  INTRODUCTION 

This  chapter  will  provide  a  qualitative  analysis  of 
the  42  cases  of  protests  of  contracts  awarded  using  a 
trade-off  process,  which  the  Comptroller  General  decided 
during  calendar  years  1998  and  1999.  First,  the  chapter 
presents  summary  statistics  for  the  eight  categories  of 
sustaining  elements  observed  among  the  cases  examined. 

Then,  it  presents  a  thorough  discussion  of  each  category. 
Additionally,  whereas  Palmer  [Ref. 7]  examined  all  the 
sustained  cases  for  the  single  year  studied,  this 
researcher  will  limit  discussion  to  those  cases  that  best 
portray  each  of  the  sustaining  element  categories.  As  a 
side  note,  the  researcher  observed  three  instances  in  which 
one  of  the  sustaining  elements  cited  by  the  Comptroller 
General  was  not  an  element  relevant  to  the  trade-off 
process  (e.g.,  improper  agency  statement  of  need,  improper 
agency  treatment  of  competing  offerors,  etc.).  These 
instances  will  not  be  discussed  in  this  chapter;  nor  were 
they  included  in  the  computation  of  the  statistics 
presented  in  the  next  section. 
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B .  SUMMARY  STATISTICS 

For  the  period  studied,  the  Comptroller  General's  most 
frequently  cited  reason  for  sustaining  protests  was  that 
either  the  Source  Selection  Evaluation  Board  (SSEB)  or  the 
Source  Selection  Authority  (SSA) — or  both — had  not 
adequately  documented  their  case  files  in  support  of  the 
award  recommendation  or  the  award  decision,  respectively. 
This  was  a  sustaining  element  in  17  of  the  42  cases 
analyzed,  or  40  percent  (40.48  percent). 

Considering  the  total  number  of  sustaining  elements — 71 
in  all — cited  by  the  Comptroller  General  for  all  42  cases, 
this  particular  element  was  noted  24  percent  (23.94 
percent)  of  the  time. 

Occurring  slightly  less  often  was  the  judgment  that 
the  evaluation  of  proposals  in  response  to  a  particular 
solicitation  had  not  been  conducted  in  accordance  with  the 
stated  evaluation  scheme  of  the  Request  For  Proposals 
(RFP) .  This  was  a  sustaining  element  in  14  cases,  or  33 
percent  (33.33  percent).  As  a  whole,  this  represented  20 
percent  (19.72  percent)  of  all  sustaining  elements  passed 
down  in  Comptroller  General  decisions  for  the  period 
studied. 

The  next  most  frequent  cause  of  sustained  protests  was 
an  improper  evaluation  by  the  awarding  agency  of  a 
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proposal's  cost  or  price.  This  occurred  in  12  of  the  42 
cases  studied,  or  29  percent  (28.57  percent).  The 
frequency  of  this  cause  relative  to  all  causes,  according 
to  the  researcher's  analysis,  was  17  percent  (16.90 
percent) . 

Next,  two  sustaining  elements  occurred  with  equal 
frequency.  The  first  was  that  the  awarding  agency 
improperly  evaluated  an  offeror's  past  performance  when 
conducting  trade-offs;  the  other  was  that  an  agency  failed 
to  conduct  meaningful  discussions  with  offerors,  as 
required  by  the  Federal  Acquisition  Regulation  (FAR) . 

These  sustaining  elements  each  occurred  in-  nine  cases,  or 
21  percent  (21.43  percent)  of  the  cases  analyzed. 

Another  frequently  occurring  sustaining  element  was  an 
awarding  agency' s  improper  evaluation  of  the  technical 
merit  of  an  offeror's  proposal.  This  was  a  sustaining 
element  in  seven,  or  17  percent  (16.67  percent),  of  the  42 
sustained  protests  of  contracts  awarded  using  the  trade-off 
process.  As  a  function  of  all  elements  cited  by  the 
Comptroller  General  in  sustaining  the  protests  studied, 
this  element  accounted  for  ten  percent  (9.86  percent). 

In  two  cases,  the  Comptroller  General  noted  that  an 
awarding  agency  had  improperly  evaluated  the  labor 
qualifications  of  key  personnel  proposed  by  offerors  in 


41 


particular  cases.  This  equates  to  five  percent  (4.76 
percent)  of  the  42  cases  studied  and  three  percent  (2.82 
percent)  of  all  sustaining  elements  cited  by  the 
Comptroller  General. 

Finally,  there  was  a  single  case  in  which  the  awarding 
agency  had  failed  to  adequately  disclose  the  factor 
evaluation  weights  in  the  solicitation.  This  one  case 
accounted  for  two  percent  (2.38  percent)  of  all  relevant 
cases  for  the  period  studied.  Additionally,  the  single 
occurrence  of  this  particular  sustaining  element  equates  to 
a  mere  one  percent  (1.41  percent)  of  all  elements  cited  by 
the  GAO  in  the  subject  cases. 

The  eight  sustaining  elements,  and  the  frequency  with 
which  each  occurs,  are  summarized  in  Table  6.  Figure  5 
presents  the  same  information  graphically. 
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Table  7.  Sustaining  elements  and  frequency 
Source.  Developed  by  the  researcher 


Sustaining  Element 

Frequency 

Percent  of 

Total  Cases 

Selection  Decision  not 

Adequately  Documented 

17 

40% 

^Evaluation  of  Proposals 

Inconsistent  with  RFP 

14 

33% 

Improper  Agency  Evaluation 

of  Cost/Price 

12 

29% 

Improper  Agency  Evaluation 

of  Past  Performance 

9 

21% 

Agency  Improperly  Failed  to 

Conduct  Meaningful 

Discussions 

9 

21% 

Improper  Agency  Evaluation 

of  Technical  Merit 

7 

17% 

Improper  Agency  Evaluation 

of  Labor  Qualifications 

2 

5% 

Failure  to  Adequately 

Disclose  Factor  Evaluation 

Weights  in  Solicitation 

1 

2% 
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Legend: 


A  -  Selection  decision  not  adequately  documented 
B  -  Evaluation  of  proposals  inconsistent  with  RFP 
C  -  Improper  agency  evaluation  of  cost/price 
D  -  Improper  agency  evaluation  of  cost/price 

E  -  Agency  improperly  failed  to  conduct  meaningful  discussions 
F  -  Improper  agency  evaluation  of  technical  merit 
G  -  Improper  agency  evaluation  of  labor  qualifications 

H  -  Failure  to  adequately  disclose  factor  evaluation  weights  in  solicitation 

j>\ 

r  ■  \ 

'  Fivgure  3.  Sustaining  elements  as  a  percent  of  total 

cases  analyzed 

Source.  Developed  by  the  researcher 


It  is  interesting  to  note  that,  in  23  cases,  (54.76 


percent)  the  protest  was  sustained  based  on  a  single 
sustaining  element  and  that,  in  six  cases,  GAO  sustained 
the  protest  citing  three  or  more  sustaining  elements.  In 
fact,  the  Comptroller  General  cited  four  or  more  sustaining 
elements  in  three  decisions  (7.14  percent).  This  is  not 
terribly  significant  since  it  takes  only  one  proven  flaw 
for  GAO  to  sustain  the  protest,  but  it  does  indicate  that  a 
small  percentage  of  cases  are  plagued  by  many  problems. 
Figure  6  graphically  presents  the  distribution  of  the 
various  combinations  of  sustaining  elements. 


Figure  4.  Sustaining  Element  Combination 
Source.  Developed  by  the  researcher 


C.  SUSTAINING  ELEMENT  CATEGORIES 

1 .  Selection  Decision  Not  Adequately  Documented 

The  following  subsections  will  present  two  sustained 
protests  of  contracts  awarded  using  a  trade-off  process  in 
which  the  selection  decision  was  not  adequately  documented. 
This  failure  to  document  the  trade-offs  made  in  the 
decision  process  was  a  fatal  flaw,  which  ultimately  led  to 
the  offeror's  protest  being  sustained.  Additionally,  the 
researcher  will  make  recommendations  for  Federal 
contracting  officers  to  consider  in  order  to  mitigate  the 
risk  of  suffering  sustained  protests  due  to  inadequately 
documented  selection  decisions. 

a.  Biospherics  Incorporated 

Biospherics  Incorporated;  B-278508.4,  B-218508 .5 , 
B-278508 . 6;  Comptroller  General  of  the  United  States;  6 
October  1998,  (Biospherics)  is  a  protest  in  which  agency 
failure  to  adequately  document  its  selection  decision  was 
flawed.  This  flaw  led  GAO  to  conclude  that  it  was  not 
possible  to  determine  the  reasonableness  of  the  agency 
evaluation  upon  which  the  award  decision  was  made. 
Biospherics  protested  the  Agency  for  Health  Care  Policy  and 
Research  (AHCPR) ,  Department  of  Health  and  Human  Services 
award  of  a  contract  to  Logistics  Applications,  Inc,  (LAI) 
for  the  operation  of  a  publication  clearing  house.  The 
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Request  For  Proposals  (RFP)  contemplated  the  award  of  a 
cost-plus-fixed-fee  contract  for  the  base  period  and  four 
one-year  option  periods.  The  RFP  required  the  contractor 
to  store  and  distribute  AHCPR  publications;  to  maintain  and 
manage  AHCPR' s  automated  mailing/inventory  control  systems; 
and  to  manage  the  storage  and  shipping  of  AHCPR  exhibits. 
Award  was  to  be  made  to  the  best  advantage  of  the 
Government,  cost  and  other  factors  considered. 

The  following  technical  evaluation  factors  were 
contained  in  the  RFP:  1)  understanding  the  problem;  2) 
technical  approach;  3)  management  plan;  4)  key  personnel; 
and  5)  facilities.  Each  offer  was  to  be  evaluated,  and  a 
recommendation  concerning  technical  acceptability  was  to  be 
made  by  a  peer  review  technical  committee.  Offers 
determined  to  be  technically  acceptable  would  then  be 
evaluated  for  past  performance.  The  RFP  stated  that 
technical  merit  was  a  paramount  consideration  and  that  cost 
would  become  a  significant  factor  only  if  two  or  more 
proposals  were  determined  to  be  technically  equal. 

A  peer  review  panel  consisting  of  three 
individuals  evaluated  proposals  submitted  by  three  firms. 
All  three  firms  were  included  in  the  competitive  range,  and 
the  panel  held  discussions  that  focused  on  technical  and 
cost  issues.  The  offerors  submitted  final  proposal 
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revisions.  However,  the  agency  made  no  adjustments  to  any 
of  their  technical  scores;  nor  did  it  produce  any 
documentation  reflecting  an  analysis  of  the  revised 
proposals.  AHCPR  selected  LAI  as  the  most  advantageous 
offeror  since  it  submitted  the  highest-technically-rated, 
lowest-proposed-cost  proposal. 

Biospherics  protested,  raising  two  challenges  to 
AHCPR' s  evaluation  of  its  proposal:  first,  that  its 
proposal  should  have  been  rated  technically  superior  to 
LAI's;  and  second,  that  the  agency  decided  to  award  a 
contract  to  LAI  as  a  technically  superior  offeror.  For  the 
purposes  of  this  discussion,  the  two  challenges  will  be 
treated  as  one — that  Biospheric's  proposal  was  technically 
superior  and  should  have  received  the  award. 

The  agency  countered  by  stating  that  Biospheric' s 
final  proposal  revision  represented  an  improvement  in  its 
technical  proposal,  but  not  enough  to  be  considered 
technically  equal  with  LAI.  When  GAO  asked  the  agency  to 
point  out  where  the  record  documented  a  comprehensive  look 
at  the  final  results  of  the  technical,  cost  and  past 
performance  evaluations,  the  agency  responded  by  asserting 
that  the  source  selection  memorandum  represented  the 
comprehensive  assessment  of  those  factors.  The  agency  also 
stated  that  the  source  selection  memorandum  directly 
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reflected  a  comprehensive  discussion  of  the  relative 
technical  strengths  and  weaknesses  of  the  three  proposals, 
although  that  discussion  was  not  recorded  in  writing. 

GAO  repeatedly  states  in  its  write-up  of  protest 
decisions  that  it  will  not  reevaluate  proposals,  but, 
rather,  will  examine  the  record  to  determine  whether  the 
agency's  judgment  was  reasonable  and  in  accordance  with  the 
RFP's  stated  evaluation  criteria. 

Further,  it  is  a  basic  principle  of  GAO  that, 
while  adjectival  ratings  and  point  scores  are  useful  as 
guides  to  decision-making,  they  generally  are  not 
controlling.  Instead,  they  must  be  supported  by  reasoned 
documentation  of  the  relative  differences  between  proposals 
and  the  basis  for  the  selection  decision. 

The  lack  of  reasoned  documentation  was  the  fatal  flaw  in 
this  case.  There  was  no  discussion  regarding  the  relative 
strengths  and  weaknesses  of  the  proposals  and  no  post¬ 
discussion  narrative  supporting  the  agency's  conclusion. 
Because  of  this  lack  of  documentation,  GAO  found  it  could 
not  determine  the  reasonableness  of  the  agency's  selection 
decision.  The  Comptroller  General  sustained  the  protest  on 
these  grounds  and  recommended  that  AHCPR  reevaluate  the 
proposals,  document  its  evaluation  and  make  a  new  selection 
decision.  [Ref.  8] 
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b.  Si -Nor  Incorporated 

Si-Nor  Incorporated;  B-282064,  B-282064.2; 
Comptroller  General  of  the  United  States;  25  May  1999r  (Si- 
Nor)  is  a  protest  in  which  the  agency  record  does  not 
establish  that  the  trade-off  decision  was  reasonable.  Si- 
Nor  protested  the  Air  Force  award  of  a  fixed-price 
requirements  contract  for  an  eight-month  base  period,  with 
four  one-year  options.  The  contract  was  for  refuse 
collection  and  disposal  services  at  Hickam  Air  Force  Base, 
Hawaii,  with  award  to  be  made  on  a  best-value  basis. 

The  RFP  required  no  technical  proposals;  rather, 
each  offeror  was  required  to  submit  financial  references 
and  at  least  three  past  performance  references  for  service 
contracts  of  a  similar  size  in  which  it  was  the  prime 
contractor.  Proposals  were  to  be  evaluated  based  on:  1) 
past  performance/experience;  and  2)  price/cost,  which  were 
approximately  equal  in  weight.  Past  performance  and 
experience  were  also  to  be  approximately  equal  in  weight, 
with  past  performance  to  be  assessed  to  determine  the 
offeror's  relative  reliability  to  perform  the  contract 
requirements.  Experience  was  to  be  evaluated  to  assess  the 
offeror's  experience  in  performing  work  on  similar  types  of 
service  contracts. 
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Of  the  five  offers  received,  the  Air  Force 
determined  that  the  proposal  from  U.S.  Eagle,  Incorporated 
(Eagle)  represented  the  best  value  to  the  Government  based 
on  its  record  of  outstanding  past  performance-  The  Air 
Force  believed  that  Eagle  was  worth  the  nearly  $900,000 
price  premium  over  Si-Nor's  proposal. 

Si-Nor  protested  on  the  basis  that  the  agency 
failed  to  document  why  any  advantage  reflected  in  Eagle's 
past  performance  rating  warranted  the  high  price  premium. 

In  its  judgment,  GAO  cited  its  ruling  in  the 
matter  of  Dyncorp;  B-245289 ,  B-245289.2;  Comptroller 
General  of  the  United  States;  December  23,  1991,  in  which 
it  determined  that:  Price/technical  trade-offs  may  be  made, 
and  the  extent  to  which  one  may  be  sacrificed  for  the  other 
is  governed  only  by  the  tests  of  rationality  and 
consistency  with  the  established  evaluation  criteria. 

Further,  in  the  matter  of  TRW,  Inc.;  B-234558; 
Comptroller  General  of  the  United  States;  21  June  1989;  GAO 
stated: 

In  deciding  between  competing  proposals,  the 
propriety  of  such  a  trade-off  turns  not  only  on 
the  difference  in  technical  scores  or  ratings  per 
se,  but  on  whether  the  selection  officials 
judgment  concerning  the  significance  of  that 
difference  was  reasonable  and  adequately 
justified  in  light  of  the  RFP  evaluation  scheme. 
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Based  on  these  prior  rulings  and  the  record  of 
this  particular  case,  GAO  decided  in  favor  of  the 
protestor.  The  Comptroller  General  ruled  that,  while  the 
record  clearly  states  the  agency's  conclusion  that  Eagle's 
past  performance  rating  was  worth  the  price  premium,  it 
includes  no  documentation  indicating  why  its  past 
performance  rating  would  outweigh  the  additional  price. 

The  Comptroller  General  recommended  that  the  Air  Force 
perform  and  document  a  proper  trade-off  analysis.  [Ref.  9] 
c.  Element  Summary 

The  two  cases  discussed  above  are  typical  of  the 
remainder  of  the  protests  sustained  during  the  period 
studied  that  contained  this  particular  sustaining  element. 

Contracting  officers  must  not  only  perform  a 
trade-off  analysis  that  incorporates  sound  business 
judgments,  but  they  must  also  adequately  support  their 
decisions  with  reasoned  documentation.  This  documentation 
must  include  the  benefits  associated  with  awarding  a 
contract  to  offerors  whose  proposals  represent  a  price 
premium  with  respect  to  other  technically  acceptable 
offers . 

2 .  Evaluation  of  Proposals  Inconsistent  with  RFP 

The  second  most  frequently  observed  sustaining  element 
was  the  evaluation  of  proposals  in  a  manner  that  deviated 
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from  the  stated  evaluation  scheme  of  the  RFP.  This  flaw  in 


the  evaluation  process  creates  a  bias  in  the  selection 
decision  that,  although  it  may  be  unintentional,  is  most 
certainly  detrimental  to  the  Government's  reputation  as  an 
even-handed  partner  in  its  business  dealings.  The  two 
cases  discussed  below  contained  this  category  of  sustaining 
element,  although,  as  in  previously  discussed  cases,  it  was 
not  the  only  sustaining  element  in  the  protest. 

a.  Marquette  Medical  Systems ,  Inc. 

Marquette  Medical  Systems ,  Inc.;  B-211821.5,  B- 
271821.7;  Comptroller  General  of  the  United  States;  29 
April  1999,  (Marquette)  is  a  protest  in  which  the  Federal 
agency  improperly  deviated  from  its  solicitation  evaluation 
criteria  in  assessing  competing  proposals.  Marquette 
protested  the  Army' s  award  of  a  contract  to  Hewlett-Packard 
Company  for  a  Cardiology  Medical  Information  System  (CMIS) 
to  replace  the  Department  of  Defense's  (DoD)  existing 
computerized  medical  information  system. 

The  RFP  anticipated  the  award  of  a  fixed-price 
contract  for  a  base  period  of  one  year,  with  options  for 
seven  additional  years.  The  solicitation  requested 
proposals  for  providing  hardware,  software,  documentation, 
training,  support  and  maintenance  services  relating  to 
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storage,  and  retrieval  and  interpretation  of 
electrocardiograms  (ECG) .  The  RFP  stated  that  there  were 
950  ECG  carts  in  the  existing  system  and  invited  offers 
based  upon  either  replacing  or  upgrading  the  cart 
inventory. 

Award  was  to  be  made  to  the  responsible  offeror 
whose  proposal,  conforming  to  the  solicitation,  was 
determined  to  be  the  best  overall  value,  price  and  other 
factors  considered.  The  technical  evaluation  criteria,  in 
descending  order  of  importance  (factors  three  and  four  were 
comparatively  equal)  were:  1)  quality  of  technical 
approach;  2)  contract  management;  3)  understanding  of 
overall  contract  requirements;  (4)  past  performance  and 
relevant  experience;  and  (5)  key  personnel  qualifications. 
The  Army  received  three  proposals  and  ultimately  determined 
that  Hewlett-Packard  offered  the  best  value. 

Marquette  protested  the  award  decision,  alleging 
that  the  agency's  evaluation  of  proposals  was  unreasonable 
and  inconsistent  with  the  RFP' s  evaluation  scheme.  A 
review  of  the  record  shows  that  the  contracting  officer 
considered  the  cost  of  purchasing  new  ECG  carts  over  the 
potential  eight-year  life  of  the  contract.  The  contracting 
officer  reported  that,  since  Hewlett-Packard  proposed 
replacing  the  ECG  carts  while  Marquette  offered  to  upgrade 
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them,  he  "normalized"  the  two  offers  by  deducting  a  certain 
amount  from  Hewlett-Packard's  offer  and  adding  on  to  the 
cost  of  Marquette's  offer.  The  contracting  officer  intended 
this  "normalization"  to  account  for  the  minimal  need  to 
acquire  new  carts  during  the  contract  life  for  the  Hewlett- 
Packard  offer,  as  well  as  the  additional  cost  of  replacing 
carts  over  the  contract  period  if  Marquette's  proposal  were 
accepted. 

The  flaw  in  the  contracting  officer's  logic,  as 
GAO  noted,  was  that,  while  the  RFP  indicated  that  a  price 
analysis  would  be  conducted,  it  gave  no  indication  that  the 
analysis  would  involve  deducting  the  price  for  performing 
any  part  of  the  work  (and  especially  not  the  price  for  a 
major  component  such  as  the  ECG  carts)  from  an  offeror's 
total  price  in  an  attempt  to  "normalize"  the  competition. 

In  fact,  as  the  Comptroller  General  correctly  noted  in  the 
matter  of  Sisters  of  Charity  of  the  Incarnate  World ;  B- 
251799  et.al.;  Comptroller  General  of  the  United  States ;  4 
May  1993:  "where  the  contract  is  fixed  price,  an  agency  is 
not  permitted  to  make  adjustments  for  cost  elements,  since 
the  fixed  price  is  the  price  the  awardee  is  obligated  to 
perform  at  and  the  price  the  Government  is  bound  to  pay." 

The  Comptroller  General  sustained  this  protest 
based  on  the  fact  that  once  offerors  are  informed  of  the 
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criteria  against  which  their  proposals  will  be  evaluated, 
the  agency  must  adhere  to  those  criteria  or  inform  all 
offerors  of  any  significant  changes  made  in  the  evaluation 
scheme.  Accordingly,  GAO  recommended  that  the  Army 
reconsider  whether  the  RFP' s  current  evaluation  scheme 
represented  its  needs  and,  if  so,  to  reevaluate  proposals. 
[Ref .10] 


b.  Opti-Lite  Optical 

Matter  of  Opti-Lite  Optical ;  B-281693 ; 

Comptroller  General  of  the  United  States;  22  March  1999, 
(Opti-Lite)  is  a  protest  in  which  the  award  determination 
was  improperly  based  on  a  comparison  of  total  technical  and 
price  point  scores  under  a  solicitation  that  did  not 
provide  for  such  a  point-based  trade-off.  Opti-Lite 
protested  the  Department  of  Veterans  Affairs  (VA)  award  of 
an  indefinite-delivery,  indefinite-quantity  requirements 
contract  to  Classical  Optical  Laboratories  (Classical)  for 
commercial  item  prescription  eyeglasses  and  services  for  VA 
beneficiaries . 

The  solicitation  provided  for  award  to  the 
responsible  offeror  whose  conforming  offer  was  most 
advantageous  to  the  Government.  The  stated  technical 
evaluation  criteria  consisted  of:  1)  methodology  of 
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approach;  2)  personnel  qualifications;  and  3)  past 
performance.  The  solicitation  further  provided  that 
technical  and  past  performance  combined  were  approximately 
equal  in  weight  to  price.  Of  the  six  proposals  submitted. 
Classical's  was  selected  as  being  most  advantageous  to  the 
Government. 

Opti-Lite  protested  the  VA' s  decision,  alleging, 
among  other  things,  that  the  agency's  decision  not  to  allow 
alternate  pricing  proposals  was  a  major  change  requiring 
the  agency  to  again  request  final  proposal  revisions.  The 
Comptroller  General's  discussion  of  this  protest  revolves 
around  two  issues:  a  mechanical  application  of  the 
price/technical  trade-off  methodology;  and  the  fact  that 
the  contracting  officer  apparently  made  no  qualitative 
assessment  of  the  proposals  as  required  by  the  evaluation 
scheme  stated  in  the  RFP .  In  sustaining  the  protest,  the 
Comptroller  General  stated  that  the  award  determination  was 
based  entirely  on  a  comparison  of  total  technical  and  price 
point  scores  under  a  solicitation  that  did  not  provide  for 
such  a  point-based  trade-off.  [Ref.  11] 

c .  Element  Summary 

These  sustained  protests  are  typical  of  the 
remaining  twelve  that  cited  as  a  sustaining  element  an 
agency's  deviation  from  the  evaluation  scheme  stated  in  the 
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RFP.  Where  an  agency  has  been  unwilling  or  unable  to 
adhere  to  the  prescribed  method,  the  Comptroller  General 
has  sided  with  the  protestor  in  every  instance.  The  lesson 
for  contracting  officers  is  that  they  must  abide  by  the 
stated  evaluation  criteria  of  an  RFP,  or,  at  the  very 
least,  advise  all  offerors  of  a  required  deviation  in  order 
to  allow  them  to  revise  their  proposals.  The  clear  and 

f 

convincing  penalty  for  doing  otherwise  is  to  invite  the 
risk  of  a  sustained  protest. 

3 .  Improper  Agency  Evaluation  of  Cost/Price 
This  subsection  presents  two  cases  in  which  an 
agency's  improper  evaluation  of  cost/price  resulted  in 
protest  sustainment.  Contracting  officers  must  determine 
not  only  that  a  proposal's  cost  is  fair  and  reasonable,  but 
also  what  a  contract  should  cost,  assuming  reasonable 
economy  and  efficiency. 

a.  Tidewater  Construction  Corporation 
Tidewater  Construction  Corporation ;  B-278360 ; 
Comptroller  General  of  the  United  States;  20  January  1998 , 
(Tidewater)  is  a  protest  in  which  the  agency  failed  to 
perform  a  proper  cost  realism  analysis.  Its  only 
evaluation  of  the  offerors'  proposals  was  a  comparison 
between  the  proposed  costs  for  various  work  items  and  the 
government  estimate.  Tidewater  protested  the  U.S.  Navy 
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award  of  a  cost-plus-award-fee  contract  for  the 
construction  of  a  crane  rail  interconnect  system  to 
Pizzagalli  Construction  Company. 

The  RFP  evaluation  scheme  provided  for  award  on  a 
best-value  basis  in  which  the  technical  factors  combined 
were  to  be  more  important  than  cost.  Proposals  were  to  be 
based  on  a  60-percent  design  of  the  project.  The  RFP  did 
not  request  detailed  cost  data,  but  did  advise  offerors 
that  supporting  documentation  for  large  elements  of  cost 
would  be  requested  during  discussions. 

The  Navy  hired  an  architect/engineer  to  prepare 
cost  estimates  tracking  35-percent,  60-percent,  and  100- 
percent  design  of  the  project  in  order  to  assist  in  the 
cost  evaluation.  All  proposals  received  were  included  in 
the  competitive  range.  Following  discussions,  offerors 
submitted  final  proposal  revisions  incorporating,  among 
other  things,  the  100-percent  design  of  the  project.  The 
Navy' s  cost  realism  evaluation  consisted  solely  of 
comparing  each  offeror's  target  cost  to  the  government 
estimate . 

Tidewater  protested  the  Navy's  evaluation  of  cost 
realism  and  alleged  that  the  government  estimate  was 
defective.  Defense  Federal  Acquisition  Regulation 
Supplement  sections  215.801  and  215.805-70  require  that, 
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whenever  a  cost  reimbursement  contract  is  anticipated,  a 
cost  realism  analysis  be  performed  to  determine  if  the 
overall  costs  in  an  offeror's  proposal:  1)  are  realistic 
for  the  work  to  be  performed;  2)  reflect  a  clear 
understanding  of  the  requirements;  and  3)  are  consistent 
with  the  various  elements  of  the  offeror's  technical 
proposal.  In  considering  this  protest,  GAO  noted  that  its 
review  of  an  agency' s  cost  evaluation  is  limited  to 
determining  whether  the  evaluation  was  reasonably  based  and 
was  not  arbitrary. 

In  sustaining  this  protest,  the  Comptroller 
General  found  that  the  Navy,  by  determining  cost  realism 
based  solely  on  a  comparison  between  the  offeror's  target 
costs  and  the  government  estimate,  acted  improperly.  GAO 
further  stated  that  the  agency' s  evaluation  was  a  price 
analysis  intended  only  to  ensure  a  fair  and  reasonable 
price  and  could  not  reasonably  be  used  to  ascertain  the 
cost  realism  of  a  cost  reimbursement  proposal.  The  reason 
for  this  is  that  the  Government  will  be  liable  for  the 
contractor's  allowable  and  allocable  costs,  not  the 
proposed  costs.  A  cost  realism  analysis  allows  the 
Government  to  determine  what  the  contract  should  cost, 
assuming  reasonable  economy  and  efficiency.  GAO 
recommended  that  the  Navy  obtain  cost  data  sufficient  to 
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perform  a  proper  cost  realism  analysis  and  make  a  new 
source  selection.  [Ref.  12] 

b.  E.L.  Hamm  &  Associates ,  Inc. 

E.L.  Hamm  &  Associates ,  Inc.;  B-280766.3; 
Comptroller  General  of  the  United  States ;  12  April  1999, 
(Hamm)  is  a  protest  where  the  agency  did  not  perform  a 
proper  cost  realism  evaluation.  Hamm  protested  the  U.S. 
Army  award  of  a  cost-plus-award-fee  contract,  for  a  base 
period  with  four  option  years,  for  storage  and  warehouse 
services  at  the  Army  Training  Support  Center  (ATSC) ,  Fort 
Eustis,  Virginia. 

The  RFP  requested  that  offerors  submit  separate 
technical /management  and  cost  proposals  corresponding  to 
four  evaluation  factors:  1)  technical;  2)  management;  3) 
past  performance;  and  4)  cost.  The  Army  determined  that  the 
proposal  submitted  by  Communication  Technologies,  Inc. 
(Comtek)  represented  the  best  overall  value  to  the 
Government . 

Of  the  proposals  received,  six  were  included  in 
the  competitive  range.  Discussions  were  conducted  and 
final  proposal  revisions  requested  from  all  six  offerors. 
The  evaluators  concluded  that  the  added  benefits  offered  by 
the  technically  superior,  highest-ranked  proposal  were  not 
worth  the  cost  premium  and,  therefore,  recommended  award  to 
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Comtek  as  the  offeror  with  the  next  highest-ranked, 
technically  superior  offer  with  the  lowest  evaluated  cost. 
Hamm  filed  a  protest  alleging  that  the  agency's  cost 
realism  evaluation  was  flawed,  partly  because  Comtek's 
proposed  overhead  rates  were  unrealistically  low. 

The  record  does  not  justify  the  agency's 
determination  that  Comtek' s  proposed  overhead  rate  was 
realistic.  The  contracting  officer's  rationale  for 
accepting  this  rate,  which  is  significantly  lower  than  the 
one  approved  by  the  Defense  Contract  Audit  Agency,  is 
unfounded.  Indeed,  the  contracting  officer  stated  that  the 
agency  accepted  Comtek's  proposed  rate  without  any  cost 
realism  evaluation.  For  these  reasons,  the  Comptroller 
General  found  that  the  agency's  actions  prejudiced  Hamm 
and,  therefore,  sustained  the  protest.  Among  its 
recommendations,  the  GAO  stated  that  the  agency  must 
perform  a  proper  cost  evaluation,  including  a  cost  realism 
analysis.  [Ref.  13] 

c.  Element  Summary 

These  findings  by  the  Comptroller  General  point 
to  the  importance  of  a  proper  cost /price  evaluation.  The 
Federal  Acquisition  Regulation  section  15.305  states  that, 
when  contracting  on  a  fixed-price  type  basis,  adequate 
competition  will  usually  establish  price  reasonableness. 
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For  cost  reimbursement  contracts,  evaluation  must  include  a 
cost  realism  analysis  to  determine  not  only  the  cost 
realism  of  the  offeror's  proposal,  but  also  the  cost  the 
Government  should  realistically  expect  to  pay,  assuming 
reasonable  economy  and  efficiency.  [Ref.  2:15.305(1)] 

Not  only  must  contracting  officers  determine 
cost/price  reasonableness — through  either  competition  or  a 
cost  realism  analysis — but  they  must  also  document  this 
determination.  This  goes  back  to  the  researcher's  most 
commonly  observed  sustaining  element — that  is,  performing  a 
cost  realism  analysis  without  documenting  the  results  is 
effectively  the  same  as  not  performing  the  analysis  in  the 
first  place. 

4 .  Improper  Agency  Evaluation  of  Past  Performance 

The  two  cases  discussed  below  involve  protests  that 
were  sustained  solely  because  the  contracting  agency 
improperly  evaluated  the  offeror's  past  performance.  Aside 
from  the  two  cases  presented,  this  sustaining  element  was 
cited  by  the  Comptroller  General  in  seven  other  cases  among 
the  42  studied. 

a.  U.S.  Property  Management  Service  Corporation 

U.S.  Property  Management  Service  Corporation ;  B- 
278727 ;  Comptroller  General  of  the  United  States;  6  March 
1998,  (U.S.  Property)  is  a  protest  in  which  improper  agency 
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evaluation  of  past  performance  was  the  basis  for  a 
sustained  argument.  U.S.  Property  protested  the  Department 
of  Housing  and  Urban  Development  (HUD)  award  of  a  small 
business  set-aside  contract  for  real  estate  asset 
management  services  for  single  family  properties  owned  by 
HUD  in  its  District  of  Columbia  office  jurisdiction.  HUD 
determined  that  a  proposal  by  Ernie  Stefkovic  Associates, 
Inc.  (ESA)  represented  the  best  value  to  the  Government. 

The  RFP  provided  for  a  cost/technical  trade-off 
and  stated  that  the  combined  weight  of  the  technical 
evaluation  factors  was  more  significant  than  price.  The 
RFP  listed  the  technical  evaluation  factors,  with  their 
relative  weights,  to  be  scored  on  a  100-point  scale,  as 
follows : 

•  Prior  management  experience  30  points 

•  Past  performance  25  points 

•  Management  capability  25  points 

•  Office  location (s)  20  points 

Offerors  were  required  to  supply  references  for 
recent,  relevant  contract  work  under  the  prior  management 
experience  and  past  performance  factors.  The  incumbent 
contractor  did  not  submit  a  proposal  since  it  was  not  a 
small  business.  The  presidents  of  U.S.  Property  and  ESA 
were  former  managers  of  the  incumbent  who  formed  their  own 
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property  management  company  in  1994.  Due  to  a  dispute 
between  the  principals,  these  two  individuals  broke  away 
from  this  company.  The  president  formed  ESA,  while  the  vice 
president  formed  U.S.  Property,  for  the  purpose  of 
submitting  independent  proposals  in  response  to  HUD's  RFP. 

Since  the  corporations  were  newly  formed,  the 
presidents  of  both  ESA  and  U.S.  Property  drew  upon  their 
shared  experience  with  the  incumbent  contractor  to 
establish  their  qualifications  under  the  prior  management 
experience  and  past  performance  factors. 

HUD  included  12  proposals  in  its  competitive 
range,  including  ESA'  s  and  U.S.  Property's.  After  written 
discussions,  it  further  narrowed  the  field  to  five 
proposals,  still  including  ESA’s  and  U.S.  Property's.  The 
record  shows  that  ESA's  proposal  was  rated  higher  than  U.S. 
Property's,  primarily  due  to  the  prior  management 
experience  factor.  The  Technical  Evaluation  Panel  (TEP) 
downgraded  U.S.  Property's  proposal  because  it  was  a  newly 
formed  corporation  that  lacked  independent  corporate 
experience.  The  TEP  did  not  similarly  question  ESA's  lack 
of  independent  corporate  experience. 

In  its  revised  proposal,  U.S.  Property  provided 
evidence  that  it  had  recently  acquired  independent 
corporate  experience,  having  been  awarded  two  contracts  for 
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property  management  services.  The  TEP  subsequently 
increased  the  protestor's  prior  management  experience  score 
slightly.  Ultimately,  the  TEP  awarded  significantly  higher 
scores  for  prior  management  experience  and  past  performance 
to  ESA  than  it  did  to  U.S.  Property. 

Based  on  the  TEP's  recommendation  and  the  source 
selection  authority's  belief  that  ESA  possessed  more 
corporate  experience  than  U.S.  property,  HUD  awarded  the 
contract  to  ESA.  U.S.  Property  protested  on  the  grounds 
that  the  agency  had  conducted  an  unequal  evaluation  with 
respect  to  prior  corporate  experience. 

The  Comptroller  General's  review  of  the  record 
confirmed  that  HUD  did  not  evaluate  proposals  equally  under 
either  the  prior  corporate  experience  or  the  past 
performance  factor.  The  TEP  consistently  downgraded  the 
protestor's  proposal  in  these  two  areas,  yet  could 
demonstrate  nothing  in  the  record  to  explain  why  the  same 
judgment  did  not  apply  to  ESA,  especially  since  ESA's 
proposal  reflected  no  corporate  experience  beyond  its 
principal's  shared  experience  with  the  president  of  U.S. 
Property. 

Finally,  the  Comptroller  General  judged  that 
ESA' s  high  scores  stemmed  from  the  agency' s  evaluation  of 
its  incumbent  personnel,  not  from  its  corporate  experience, 
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which  was  the  standard  by  which  it  had  evaluated  all  other 
proposals . 

Based  on  these  findings,  the  Comptroller  General 
sustained  the  protest  and  recommended  that  the  agency 
reevaluate  the  final  proposal  revisions.  [Ref.  14] 

Jb.  Nova.  Group,  Inc. 

Nova  Group,  Inc.;  B-282947;  Comptroller  General 
of  the  United  States;  15  September  1999,  (Nova)  is  a 
protest  in  which  the  agency  improperly  downgraded  an 
offeror' s  past  performance  based  merely  on  that  offeror' s 
history  of  contract  claims.  Nova  protested  the  U.S.  Navy's 
award  of  a  fixed-price  construction  contract  for  two 
projects,  with  options  for  eight  additional  projects,  at 
the  Naval  Station,  Pearl  Harbor,  Hawaii.  The  Navy  awarded 
to  Healy  Tibbitts  Building,  Inc.  as  the  offeror  submitting 
the  best  value  proposal. 

The  solicitation  stated  that  its  trade-off 
evaluation  plan  gave  equal  weight  to  technical  evaluation 
and  price.  The  RFP  listed  the  technical  evaluation 
factors,  in  descending  order  of  importance,  as  follows: 

•  Construction  management  plan 

•  Experience/past  performance 

•  Experience  of  key  personnel 

•  Small  business  subcontracting  plan 
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With  regard  to  the  experience/past  performance 
factor,  Nova's  proposal  was  rated  satisfactory,  rather  than 
outstanding,  because  its  proposal  had  not  clearly  addressed 
the  disposition  of  a  number  of  contract  claims.  The 
Technical  Evaluation  Board  (TEB)  determined  that  Nova's 
claims  record  did  not  demonstrate  a  general  trend  for 
outstanding  past  performance  in  terms  of  customer 
satisfaction.  Nova  protested  the  evaluation  of  its 
proposal  under  the  first  two  evaluation  factors. 

Discussion  of  this  protest  will  be  limited  to  the 
evaluation  of  Nova's  experience/past  performance  since  the 
Comptroller  General  denied  the  first  protest  issue.  The 
protestor  alleged  that  the  Navy  improperly  considered 
Nova's  contract  claims  as  the  basis  for  downgrading  its 
rating. 

GAO  notes  that,  absent  some  evidence  of  abuse  of 
the  contract  dispute  process,  contracting  agencies  should 
not  downgrade  an  offeror's  proposal  simply  because  it  has 
filed  claims.  GAO  continues  by  noting  the  matter  of  Amclyde 
Engineered  Products  Co.,  Inc.;  B-282271 ,  B-282271.2; 
Comptroller  General  of  the  United  States;  21  June  1999,  in 
which  it  found  that  contract  claims  constitute  remedies 
established  by  statute  and  that  firms  should  not  be 
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prejudiced  because  of  their  reasonable  pursuit  of  such 
remedies  in  the  past. 

Since  Nova's  actual  past  performance  had  been 
rated  "average  to  outstanding"  by  previous  contracting 
agencies,  the  Navy's  only  basis  for  downgrading  Nova's 
proposal  was  the  existence  of  nine  previous  contract 
claims.  The  Comptroller  General  ruled  that  the  Navy's 
evaluation  penalized  Nova  simply  for  using  the  contract 
dispute  process  and,  therefore,  sustained  the  protest. 
Consequently,  the  Comptroller  General  recommended  that  the 
Navy  reopen  discussions,  evaluate  proposals  consistent  with 
its  decision,  and  make  a  new  source  selection  decision. 

[Ref.  15] 

c.  Element  Summary 

As  the  above  two  cases  illustrate,  contracting 
officers  have  an  obligation  to  equitably  and  fairly 
evaluate  past  performance  as  an  indicator  of  an  offeror's 
ability  to  perform  a  contract  successfully,  if  the 
solicitation  calls  for  such  an  evaluation.  The 
solicitation  must  also  explain  how  offeror's  proposals 
without  relevant  past  performance  will  be  evaluated.  The 
FAR  specifically  states  that  offerors  without  a  record  of 
past  performance  will  not  be  evaluated  favorably  or 
unfavorably  on  past  performance.  [Ref.  2:15.305(2)] 
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5 .  Agency  Improperly  Failed  to  Conduct  Meaningful 
Discussions 

Biospherics ,  Inc.  and  Mechanical  Contractors ,  S.A.  are 
just  two  of  nine  cases  that  were  sustained  due  to  an 
agency' s  failure  to  properly  conduct  meaningful 
discussions.  As  the  researcher  will  show,  if  an  agency 
conducts  discussions  with  offerors,  such  discussions  must 
be  meaningful  and  must  inform  offerors  of  the  areas  their 
proposals  that  require  amplification  or  correction.  To  do 
otherwise  increases  the  risk  of  a  sustainable  protest, 
a.  Biospherics,  Inc. 

Biospherics ,  Inc.;  B-278278;  Comptroller  General 
of  the  United  States;  14  January  1998,  (Biospherics)  is  a 
protest  in  which  the  agency  conducted  prejudicially 
inadequate  and  misleading  discussions.  Biospherics 
protested  the  U.S.  Forest  Service's  award  of  a  fixed-price, 
requirements  contract  for  the  development,  implementation, 
and  operation  of  the  National  Recreation  Reservation 
Service.  The  RFP  advised  offerors  that  award  would  be 
based  on  a  cost/technical  trade-off  and  that  technical 
evaluation  factors  were  more  important  than  price.  The 
contract  was  to  be  for  a  9.5  year  period.  The  Forest 
Service  found  that  Park.net,  Inc.'s  (Park. Net)  proposal 
best  represented  overall  value  to  the  Government. 
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The  Forest  Service  held  written  and  oral 


discussions  with  three  offerors  in  the  competitive  range. 
Since  all  three  proposals  contained  unit  pricing  that  was 
higher  than  the  independent  Government  estimate,  the  agency 
advised  them  to  review  their  cost  information  to  determine 
the  best  pricing  scheme  they  could  offer.  After  more 
discussions  and  upon  reviewing  the  final  proposals  of  all 
offerors  in  the  competitive  range,  the  agency  made  award  to 
Park. Net,  citing  Biospherics'  proposed  prices  as 
unrealistically  low.  Biospherics  alleged  that  the  agency's 
discussions  regarding  Biospherics'  proposal  prices  were 
inadequate  and  misleading.  Specifically,  Biospherics 
alleged  that  the  agency  not  only  failed  to  inform  them  that 
it  viewed  Biospherics  pricing  as  unrealistically  low,  but 
also,  following  the  first  round  of  discussions,  stated  that 
the  firm  should  review  its  proposal  "for  any  additional 
savings."  The  agency  countered  by  stating  that  it  found 
Biospherics'  prices  unrealistically  low  only  after 
reviewing  the  final  proposal  revisions.  The  agency 
admitted,  however,  that  it  had  not  conducted  complete  cost 
realism  analyses  on  the  interim  proposals. 

In  this  case,  the  Comptroller  General  found  that 
the  agency  did  not  inform  Biospherics  during  discussions 
that  it  considered  its  pricing  to  be  unrealistically  low, 
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but  instead  encouraged  Biospherics  to  reduce  its  pricing  to 
a  level  that  the  agency  ultimately  found  unrealistic.  The 
Comptroller  General  ruled  that  the  Forest  Service  had 
conducted  inadequate  and  misleading  discussions  and 
recommended  that  it  reopen  discussions  with  Biospherics  and 
Park. Net,  request  new  final  proposal  revisions,  and 
reevaluate  proposals.  [Ref.  16] 

b.  Mechanical  Contractors ,  S. A. 

Mechanical  Contractors,  S.A.;  B-277916.2; 
Comptroller  General  of  the  United  States'  4  March  1998, 
(MCSA)  is  a  protest  in  which  the  agency  did  not  conduct 
meaningful  discussions.  MCSA  protested  the  award  of  a 
firm,  fixed— price  contract  by  the  Panama  Canal  Commission 
(PCC)  for  cleaning  and  painting  four  miter  gate  leaves  in 
the  Panama  Canal.  This  protest  followed  the  reevaluation 
of  proposals  subsequent  to  a  prior  sustained  protest  filed 
by  the  same  protester  in  regard  to  the  same  solicitation. 
The  PCC  determined  that  the  proposal  submitted  by  Formal 
Management  Systems,  Inc.  represented  the  best  value  to  the 
Government . 

The  RFP  specified  that  offerors  were  to  submit 
technical  proposals  and  price  schedules  and  that  technical 
proposals  were  to  be  evaluated  on  the  basis  of  two  equally 
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weighted  technical  factors:  1)  technical  approach;  and  2) 
performance  capability. 

The  evaluation  board  in  the  original  source 
selection  had  evaluated  MCSA  as  excellent  or  better  in 
three  technical  subfactors  and  had,  therefore,  not  held 
discussions  with  MCSA  on  those  particular  areas.  The 
evaluation  board  in  this  case  had  severely  downgraded 
MCSA' s  scores  in  the  same  technical  subfactors,  but  did  not 
hold  discussions  with  MCSA  regarding  the  newly  discovered 
deficiencies . 

MCSA  argued  that  the  PCC  should  have  discussed 
these  perceived  deficiencies  with  the  company  because  the 
reevaluation  went  beyond  the  scope  of  the  original  protest. 
The  Comptroller  General  agreed  and  found  that  MCSA  was 
prejudiced  by  the  agency's  failure  to  hold  discussions  on 
the  deficiencies  identified  in  the  reevaluation.  The 
contracting  officer  specifically  cited  these  deficiencies 
as  the  basis  for  his  conclusion  that  the  contract  schedule 
would  be  at  risk  if  the  contract  were  awarded  to  MCSA.  GAO 
sustained  the  protest,  but  since  performance  of  the 
contract  had  already  begun,  its  only  recommendation  was 
that  MCSA  be  reimbursed  for  its  proposal  preparation  costs 
and  the  reasonable  costs  of  filing  this  protest.  [Ref.  17] 
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c.  Element  Summary 

In  order  to  perform  fair  evaluations  with  equal 
treatment  to  all  offerors,  contracting  officers  are 
required  to  perform  meaningful  discussions  in  negotiated 
procurements  unless  the  solicitation  specifically  states 
that  award  may  be  made  without  discussions.  Meaningful 
discussions,  as  defined  by  the  FAR,  are  those  exchanges 
that  seek  to  identify  the  significant  weaknesses, 
deficiencies,  or  other  aspects  of  the  offeror's  proposal 
that  could  be  altered  or  explained  to  materially  enhance 
the  proposal's  potential  for  award.  [Ref.  2 : 15 . 306 (d) (3) ] 

6 .  Improper  Agency  Evaluation  of  Technical  Merit 
The  two  cases  in  the  following  subsections  discuss 
protests  that  were  sustained  partly  due  to  improper  agency 
evaluation  of  technical  merit.  This  particular  sustaining 
element  occurred  in  17  percent  (16.67  percent)  of  the  cases 
analyzed. 

a .  ENMAX  Corporation 

ENMAX  Corporation;  B-281965;  Comptroller  General 
of  the  United  States ;  12  May  1999,  (ENMAX)  is  a  protest  in 
which  the  agency  improperly  evaluated  a  proposal's 
technical  merit.  ENMAX  protested  the  U.S.  Air  Force  award 
of  a  fixed-price  labor  contract  to  Carlisle  Research,  Inc. 
(Carlisle)  for  engineering  services  at  Hill  Air  Force  Base, 
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Utah.  The  RFP's  evaluation  scheme  stated  that  award  would 


be  made  using  a  performance/price  trade-off,  which 
permitted  trade-off  between  price  and  the  past  performance 
evaluation  for  technically  acceptable  proposals.  Past 
performance  and  price  were  equally  important.  The  Air 
Force  selected  Carlisle  as  the  offeror  whose  proposal 
presented  the  best  combination  of  performance  risk  and 
evaluated  price. 

ENMAX  alleged  that  Carlisle's  proposal  did  not 
meet  the  minimum  technical  requirements  set  forth  in  the 
RFP,  and  that  the  Air  Force  improperly  concluded  that  the 
proposal  was  technically  acceptable.  Specifically,  the  RFP 
stated  that  proposals  shall  address  each  and  every  area  of 
each  factor.  Since  the  technical  area  is  rated  as 
pass/fail,  failure  to  address  even  a  single  factor  may 
result  in  failure  of  the  technical  proposal. 

Carlisle's  technical  proposal  did  not  address 
each  and  every  factor  of  the  technical  area,  yet  the  Air 
Force  evaluators  gave  the  proposal  a  rating  of  "pass"  in 
the  "demonstrated  experience  (2  plus  years)  in 
client/server  development  in  ORACLE,  UNIX,  NT  and 
Powerbuilder  environments . " 

GAO  notes  that  it  is  not  the  internal  source 
selection  plan,  but,  rather,  the  specific  language  in  the 
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RFP  that  determines  whether  offerors  have  been  treated 
fairly  in  an  evaluation.  In  this  case,  GAO  stated  that  the 
RFP  did  not  specify  that  failure  to  meet  any  one  of  the 
technical  areas  would  result  in  a  finding  of  technical 
unacceptability  under  the  factor  overall.  Further,  the  RFP 
required  offerors  to  address  each  and  every  area  within 
each  factor,  but  did  not  state  that  failure  to  do  so  would 

r 

automatically  result  in  failure  of  the  proposal.  So,  even 
though  the  RFP  did  not  mandate  failure  of  this  proposal 
given  the  language  of  the  solicitation  itself,  GAO 
concluded  that  it  clearly  could  have  affected  the 
evaluator' s  conclusion  that  Carlisle  was  technically 
acceptable.  Finally,  GAO  concluded  that  the  evaluators  had 
to  consider  whether  Carlisle's  failure  to  demonstrate  the 
required  experience  made  it  technically  unacceptable  under 
this  evaluation  factor.  GAO  sustained  the  protest  on  that 
basis.  The  Comptroller  General  recommended  that  the  agency 
reevaluate  Carlisle's  technical  proposal,  taking  into 
account  whether  the  proposal  should  be  considered 
acceptable  overall  given  Carlisle's  failure  to  show 
experience  in  the  stated  computing  environments.  [Ref.  18] 

Jb.  ATA  Defense  Industries ,  Inc. 

ATA  Defense  Industries ,  Inc.;  B-282511,  B- 
282511.2;  Comptroller  General  of  the  United  States;  21  July 
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1999,  (ATA)  is  a  protest  that  was  sustained  because  of 
improper  technical  evaluation  by  a  Federal  Agency.  ATA 
protested  the  award  of  a  fixed-price  contract  by  the  U.S. 
Army  for  the  Intermediate  New  Generation  Army  Targetry 
System.  The  Army  selected  Caswell  International 
Corporation  (Caswell)  as  the  offeror  whose  proposal 
represented  the  best  value  to  the  agency. 

The  RFP  identified  four  evaluation  elements:  1) 
technical;  2)  performance  risk;  3)  small  business 
utilization;  and  4)  price.  Of  these,  technical  was  the 
most  important  and  included  the  five  technical  subfactors 
listed  below  in  descending  order  of  importance: 

•  Hit  detection 

•  Overall  design  and  stability  of  the  design 

•  Battle  effects  simulator 

•  Radio  frequency  band  width 

•  Training/manual 

Of  the  subfactors,  the  first  two  were 
significantly  more  important  than  all  the  others  combined. 
ATA' s  contention  was  that  Caswell's  proposal  was  improperly 
given  an  excellent  rating  for  hit  detection,  in  violation 
of  the  Army's  evaluation  scheme.  The  Comptroller  General 
made  extensive  inquiries  into  the  technical  aspects  of  the 
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hit  detection  subfactor  and  concluded  that  the  Army  had 
indeed  violated  its  stated  evaluation  scheme  as  delineated 
in  the  solicitation.  The  Comptroller  General  sustained  the 
protest  and  recommended  that  the  Army  reopen  the 
procurement  and  reevaluate  proposals  in  accordance  with  the 
stated  evaluation  scheme.  [Ref.  19] 
c.  Element  Summary 

The  cases  discussed  above  represent  another 
problem  that  occurs  when  Federal  contracting  officers  fail 
to  evaluate  proposals  in  a  manner  consistent  with  the 
evaluation  scheme  stated  in  the  solicitation.  Whenever  an 
evaluation  team  or  source  selection  authority  deviates  from 
the  selection  criteria  stated  in  an  RFP,  the  Government  is, 
in  essence,  inviting  a  protest.  In  order  to  maintain  the 
perception  of  a  level  playing  field  for  all  prospective 
contractors,  contracting  officers  must  adhere  to  the  stated 
evaluation  scheme  and  treat  all  offerors  equally. 

7 .  Improper  Agency  Evaluation  of  Labor 
Qualifications 

The  following  two  subsections  discuss  sustained 
protests  in  which  the  agency  improperly  evaluated  the  labor 
qualifications  of  personnel  proposed  by  the  prospective 
contractors.  In  both  cases,  the  protests  were  sustained 
because  the  agencies  failed  to  strictly  adhere  to  the  RFP's 
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evaluation  scheme  in  assessing  the  qualifications  of  an 
offeror's  proposed  personnel. 

a.  Aerospace  Design  6  Fabrication ,  Inc. 

Aerospace  Design  &  Fabrication ,  Inc.;  B-278896 .2 , 
B-278896 . 3 ,  B-278896.4,  B-278896.5 ;  4  May  1998,  (ADF)  is  a 
protest  in  which  improper  agency  evaluation  of  labor 
qualifications  resulted  in  sustainment.  ADF  protested  the 
National  Aeronautics  and  Space  Administration  (NASA)  award 
of  a  contract  to  Dynacs  Engineering  Co.,  Inc.  (Dynacs)  for 
scientific,  engineering,  technical  and  administrative 
services  for  NASA's  Lewis  Research  Center.  The  RFP 
anticipated  a  competitive  procurement  limited  to  small, 
disadvantaged  businesses  and  leading  to  a  hybrid  contract 
containing  approximately  ten  percent  fixed-price  and  90 
percent  cost  reimbursement  task  orders.  The  solicitation 
set  forth  three  evaluation  factors  of  equal  importance:  1) 
mission  suitability;  2)  cost;  and  3)  relevant  experience 
and  past  performance.  The  relevant  subfactor  under  mission 
suitability  was  key  personnel  and  was  worth  one-quarter  of 
the  total  points  assigned  to  the  factor. 

ADF  argued  that  Dynacs  misrepresented  the  level 
of  commitment  from  three  of  its  proposed  key  personnel, 
leading  the  agency  to  improperly  award  Dynacs  more  credit 
in  this  evaluation  factor  than  its  proposal  merited. 
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In  reaching  its  decision,  GAO  cited  previous 
findings  of  similar  circumstances  and  stated  that  an 
agreement  to  work  for  an  offeror,  if  it  wins  the 
competition,  is  nothing  more  than  a  promise  to  negotiate 
for  employment  and  not  a  binding  commitment.  GAO  went  on 
to  state  that  if  an  offeror  falsely  states  in  its  proposal 
that  certain  key  personnel  have  committed  to  work  for  them, 
and  the  agency  relies  on  this  misrepresentation  in  its 
evaluation,  then  the  selection  decision  is  improper.  For 
this  reason,  the  Comptroller  General  found  that  ADF  was 
prejudiced  in  the  award  decision  and  sustained  the  protest. 
[Ref.  20] 

b.  Possehn  Consulting 

Possehn  Consulting ;  B-278579 .2 ;  Comptroller 
General  of  the  United  States ;  29  July  1998 ,  (Possehn)  is  a 
protest  in  which  an  agency' s  evaluation  ignored  evidence  of 
required  experience,  and  this  resulted  in  protest 
sustainment.  Possehn  protested  the  award  of  a  fixed-price 
contract  by  the  U.S.  Forest  Service  for  the  location  and 
installation  of  permanent  forest  inventory  plots  at  various 
national  forests.  Award  was  made  to  Steve  Holmes  Forestry 
as  the  offeror  whose  proposal  represented  the  best  value  to 
the  Government.  This  protest  was  the  result  of  a 
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reevaluation  of  proposals  subsequent  to  a  prior  sustained 
protest  also  brought  before  GAO  by  Possehn. 

The  RFP  identified  four  technical  factors  in 
descending  order  of  importance: 

•  Offeror's  experience  establishing  permanent 
forest  and/or  vegetation  inventory  plots 

•  Past  performance  on  similar  contracts  during  the 
previous  three  years. 

•  Experience  of  key  management  personnel. 

•  Qualifications  of  offered  field  personnel. 

The  relevant  factor  for  purposes  of  this  discussion  is  the 
final  one — qualifications  of  offered  field  personnel.  The 
agency's  record  of  its  evaluation  of  this  factor  regarding 
Possehn' s  proposal  incorrectly  stated  that  field  crew 
members  did  not  have  experience  establishing  permanent  plot 
inventory  systems.  However,  Possehn' s  final  proposal 
revision  clearly  identified  personnel  with  such  experience 
and  indicated  that  they  had  acquired  the  experience  by 
establishing  such  plots  for  the  Forest  Service. 

In  its  decision,  GAO  concluded  that  the  agency 
had  improperly  overlooked  the  experience  of  Possehn' s 
proposed  field  crew  and,  therefore,  it  sustained  the 
protest.  The  Comptroller  General  recommended  that  the 
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agency  reevaluate  Possehn's  final  proposal  revision  and 
make  an  appropriate  judgment  about  the  value  of  the 
experience  of  the  personnel  offered.  [Ref.  21] 

c.  Element  Summary 

The  two  cases  discussed  are  examples  in  which  the 
contracting  agency  improperly  evaluated  the  labor 
qualifications  of  prospective  contractors.  In  the  first 
case,  the  agency  relied  on  factual  misrepresentations  in 
the  offerors  proposal.  It  may  seem  reasonable  to  believe 
that  the  agency  could  have  done  nothing  to  prevent  the 
offeror  from  misrepresenting  itself.  However,  the  agency 
could  have  significantly  reduced  the  likelihood  of  relying 
on  a  misrepresentation  had  it  required  signed  commitment 
letters  from  key  proposed  personnel  who  were  not  yet  on  the 
offeror's  payroll. 

In  the  second  case,  the  agency  blatantly  erred 
when  it  reviewed  the  offeror's  final  proposal  revision, 
overlooking  the  precise  qualifications  it  desired  of  field 
personnel.  Here,  the  agency's  attention  to  detail  would 
have  drastically  reduced  the  possibility  of  a  protest — 
specifically  a  follow-on  protest. 
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8.  Failure  to  Adequately  Disclose  Factor  Evaluation 
Weights  in  Solicitation 

The  single  case  presented  below  was  the  only  protest 
among  those  studied  that  contained  this  particular 
sustaining  element.  The  failure  to  adequately  disclose  the 
factor  evaluation  weights  in  the  solicitation  was  one  of 
three  sustaining  elements  noted  in  the  Comptroller 
General's  decision  for  this  particular  case. 

a.  Foundation  Health  Federal  Services ,  Inc. 
Foundation  Health  Federal  Services,  Inc;  Humana 
Military  Healthcare  Services,  Inc.;  B-278189.3,  B-278189.4; 
Comptroller  General  of  the  United  States;  4  February  1998, 
(Foundation/Humana)  is  a  protest  in  which  the  agency  failed 
to  disclose  the  relative  importance  of  the  significant 
evaluation  factors.  Foundation  and  Humana  protested  the 
TRICARE  Support  Office's  (TSO)  award  of  a  fixed-price 
contract  to  Anthem  Alliance  for  Health,  Inc.  to  provide 
health  care  and  associated  administrative  services  for 
Military  Health  Services  System  beneficiaries. 

The  RFP  stated  that  the  agency  intended  to  make 
award  to  the  offeror  whose  proposal  was  most  advantageous 
to  the  Government,  price  and  other  factors  considered.  The 
Government  intended  to  award  the  contract  for  a  base  period 
with  five  one-year  options.  The  weighting  ratio  of 
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technical  area  to  cost  was  to  be  60  percent/40  percent. 
Within  the  technical  area,  the  proposal  identified  seven 
major  factors  that  included  11  technical  tasks.  Offerors 
were  required  to  respond  satisfactorily  to  all  factors  and 
subfactors;  however,  the  proposal  did  not  present  specific 
weights  for  each  factor  and  subfactor. 

Foundation  and  Humana  alleged  that  the 
solicitation  did  not  advise  offerors  of  the  evaluation 
criteria  TSO  actually  used  and  the  relative  weights  it 
assigned  to  the  factors  and  subfactors.  In  its  discussion, 
GAO  referred  to  Section  2305  of  Title  10  of  the  United 
States  Code  that  requires  solicitations  to  include  at  a 
minimum  "a  statement  of  -(i)  all  significant  factors  and 
significant  sub  factors  which  the  head  of  the  agency 
reasonably  expects  to  consider  .  .  .;  [and]  (ii)  the 

relative  importance  assigned  to  each  of  those  factors  and 
sub  factors  ..."  GAO  reasoned  that,  where  an  agency 
fails  to  advise  offerors  of  the  relative  weights  of  the 
evaluation  factors,  there  is  no  assurance  that  in  selecting 
an  offer  for  award  it  is  obtaining  the  best  value  for  the 
Government.  Finally,  the  Comptroller  General  concluded 
that,  where  the  solicitation  is  silent  as  to  the  relative 
importance  of  the  subfactors,  offerors  can  assume  that  the 
subfactors  are  weighted  approximately  equally.  In 
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sustaining  this  protest,  the  Comptroller  General  stated 
that  TSO  improperly  failed  to  adequately  disclose  the 
weights  of  the  significant  evaluation  factors  and 
subfactors  applied  in  the  evaluation.  [Ref.  22] 

b .  Element  Summary 

The  preceding  discussion  represents  the  single 
instance  of  an  agency' s  failure  to  adequately  disclose 
factor  evaluation  weights  in  a  solicitation  for  the  period 
studied.  This  does  not  diminish  its  importance,  however. 
As  GAO  pointed  out,  without  factor  evaluation  weights 
specified  in  the  solicitation,  there  is  no  assurance  that 
the  Government  is  obtaining  the  best  value  for  its  money. 

D .  INTEGRATED  ANALYSIS 

The  purpose  of  this  analysis  is  to  provide  Federal 
contracting  officers  with  the  recommendations  designed  to 
address  the  underlying  causes  of  sustained  protests 
discussed  above. 

1 .  Adequately  Support  the  Selection 

Decision  with  Reasoned  Documentation 

In  case  after  case,  whether  or  not  it  is  cited  as  a 
sustaining  element  in  the  decision,  GAO  notes  that 
personnel  are  failing  to  adequately  document  their 
decisions.  This  happens  at  all  levels  within  the  source 
selection  team.  Yet,  this  particular  deficiency  is. 
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intuitively,  the  easiest  to  correct,  particularly  since  GAO 
presumes  the  actions  of  agencies  correct  unless  proven 
otherwise.  In  this  respect,  agencies  are  given  great 
leeway  in  their  decision.  However,  with  this  latitude 
comes  the  responsibility  to  demonstrate  that  all  decisions 
are  reasonable  and  not  arbitrary. 

The  primary  way  to  ensure  proper  documentation  is 
through  adequate  training.  By  ensuring  that  personnel 
understand  the  importance  of  documenting  the  basis  of 
business  decisions,  contracting  agencies  can  reduce  the 
risk  of  sustainable  protests.  Training  can  be  incorporated 
into  the  source  selection  plan  or  conducted  on  a  just-in- 
time,  as-needed  basis.  Such  training  should  stress  not 
only  the  importance  of  documentation,  but  also  the  tools 
necessary  to  accomplish  the  task.  These  tools  might 
include  word  processing  training  as  a  foundation  upon  which 
to  build  and  can  be  tailored  to  the  individual  demands  of 
the  agency. 

Another  way  to  accomplish  adequate  documentation  might 
be  the  use  of  standardized  evaluation  forms  by  all 
personnel  involved  with  evaluating  proposals.  In  addition, 
a  specific  form  could  be  developed  for  personnel  who  may 
have  to  make  trade-off  decisions.  Such  a  form  could 
specify  the  aspect  of  the  proposal  being  considered  for  a 
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trade-off,  the  benefits  or  risk  expected,  and  any  impact  on 
risk  that  may  result. 

Adequate  staffing  could  also  reduce  the  risk  of  a 
sustained  protest  due  to  inadequate  supporting 
documentation.  In  the  current  atmosphere  of  fiscal 
restraint,  agencies  may  have  insufficient  funds  for 
training  personnel  properly.  Drawing  upon  personnel  who 
already  possess  sufficient  skills  in  evaluating  and 
documenting  source  selections  is  one  means  of  overcoming 
such  financial  obstacles.  Unfortunately,  smaller 
organizations  may  not  have  the  luxury  of  a  large  pool  of 
personnel  from  which  to  draw  experienced  team  members. 

Next,  using  management  more  aggressively  can  help 
reduce  the  risk  of  a  sustainable  protest.  By  carefully 
reviewing  the  documentation  of  source  selection  team 
members,  managers  can  verify  that  the  supporting  rationale 
for  decisions  is  both  present  and  appropriate.  Admittedly, 
this  requires  an  investment  of  the  manager's  time,  but  as 
the  cases  analyzed  in  this  thesis  have  shown,  it  would  be  a 
worthwhile  investment. 

Biospherics  demonstrated  that  if  it  were  possible  to 
rely  solely  on  the  documents  prepared  in  response  to 
protest  contentions,  an  agency's  selection  decision  might 
seldom  be  found  inadequate.  However,  the  Comptroller 
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General  has  repeatedly  stated  that,  although  the  entire 
record  will  be  considered  in  protest  resolutions,  greater 
weight  will  be  given  to  the  contemporaneous  source 
selection  materials.  If  there  had  been  any  notation  to 
support  the  relative  strengths  and  weaknesses  that 
evaluators  observed  in  the  reviewed  proposals,  the  case 
would  undoubtedly  have  turned  out  differently. 

t 

[Ref .5:p.739] 

Finally,  the  controlling  element  of  the  source 
selection  process  requires  adequate  supporting 
documentation.  FAR  part  15.608(a)(3)  requires 
documentation  to  support  the  evaluation  of  proposals, 
including:  the  basis  for  evaluation  and  analysis  of  the 
technically  acceptable  and  unacceptable  proposals;  an 
assessment  of  each  offeror's  ability  to  accomplish  the 
technical  requirement;  and  a  summary  of  findings.  The  FAR 
goes  on  to  state  that  the  basis  for  the  source  selection 
decision  and  the  relative  differences  among  proposals  must 
be  documented.  In  this  selection  decision  documentation,  an 
agency  must  describe  the  strengths,  weaknesses  and  risks  of 
the  various  proposals  in  terms  of  stated  evaluation 
criteria.  [Ref.  2:15.608 (a) (3)] 
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2.  Evaluate  Proposals  in  a  Manner  Consistent  with 
the  Solicitation 

Prospective  contractors  have  a  right  to  expect  that 
their  proposals  will  be  evaluated  in  accordance  with  the 
stated  evaluation  scheme  of  the  solicitation.  Unless  the 
solicitation  has  been  amended,  the  entire  evaluation  must 
be  based  on  the  stated  factors  throughout  the  competitive 
negotiation  process.  [Ref. 5:  p.543] 

Contracting  agencies  can  ensure  that  proposals  are 
evaluated  consistently  in  a  number  of  ways.  First,  the 
source  selection  plan  describes  the  evaluation  factors  and 
subfactors,  their  relative  importance,  and  the  methodology 
used  to  evaluate  proposals.  This  plan  also  serves  as  a 
guide  for  the  entire  source  selection  team  on  the  roles  of 
the  members  and  the  conduct  of  the  source  selection  from 
proposal  evaluation,  through  the  cost/technical  trade-off, 
to  award  decision  and  debriefing.  [Ref. 5:  p.210] 

In  Marquette,  the  solicitation  invited  offers  to 
either  replace  or  upgrade  existing  equipment.  The 
solicitation  did  not  provide  for  any  method  of 
"normalizing"  the  two  distinct  approaches.  Yet,  this  is 
exactly  what  the  source  selection  authority  did  as  part  of 
the  price  analysis  it  conducted.  Since  this  procedure  for 
comparing  proposals  was  not  specified  in  the  solicitation. 


89 


GAO  decided  in  favor  of  the  protestor.  By  specifying  in 
the  solicitation  the  procedure  for  comparing  the  two 
different  approaches,  the  source  selection  authority  could 
reasonably  have  applied  this  normalization  technique  and 
significantly  mitigated  the  risk  of  a  sustained  protest. 

Second,  it  is  essential  that  selection  personnel  have 
both  experience  in  evaluating  proposals  and  technical 
expertise  in  the  area  they  are  assigned  to  evaluate. 

Again,  in  Marquette,  not  only  did  the  source  selection 
authority  apply  a  comparative  technique  not  specified  in 
the  solicitation,  but  it  also  applied  the  technique 
incorrectly.  The  official  apparently  lacked  the  technical 
expertise  to  recognize  the  double-counting  mistake  made. 

The  next  risk-reduction  approach  for  source  selection 
teams  is  in  the  area  of  management.  As  discussed  in  the 
previous  recommendation,  by  effectively  directing  the 
members  of  the  source  selection  team  in  properly  evaluating 
proposals  and  then  verifying  the  results,  senior  personnel 
can  help  ensure  consistent  evaluations. 

Finally,  there  are  statutes  and  regulations  dictating 
that  evaluations  shall  be  based  solely  on  the  factors 
specified  in  the  solicitation.  By  adhering  to  these 
principles,  contracting  agencies  can  further  mitigate  the 
risk  of  sustainable  protests.  [Ref . 5 :p. 542] 


3.  Thoroughly  Conduct  Cost/Price  Realism  Analyses 
when  Required ,  and  Always  Evaluate  the  Total 
Cost/Price 

Improper  agency  evaluation  of  cost/price  was  a  factor 
in  nearly  one-third  of  all  sustained  protests  for  the 
period  studied.  In  Tidewater ,  the  Navy's  cost  realism 
evaluation  consisted  solely  of  comparing  each  offeror's 
target  cost  with  the  Government  estimate.  Since  this  was  a 
cost-reimbursement  contract,  a  cost  realism  analysis  was 
required  to  determine  if  the  overall  costs  were  realistic 
for  the  work  to  be  performed  and  reflected  a  clear 
understanding  of  the  agency's  requirements.  In  the 
planning  stage  of  the  acquisition,  the  agency  could  have 
considered  the  risk  associated  with  not  conducting  a  cost 
realism  analysis  and  made  allowances  in  its  source 
selection  plan.  Specifically,  the  agency  could  have 
engaged  the  services  of  the  Defense  Contract  Audit  Agency 
( DCAA)  to  perform  a  pre-award  survey.  Such  forethought  by 
the  source  selection  team  involved  in  Hamm  could  have 
averted  that  unfavorable  decision.  In  that  case,  the 
contracting  officer  accepted  a  proposed  overhead  rate  that 
was  significantly  lower  than  DCAA' s  approved  rate.  This 
fatal  flaw  could  have  been  avoided  if  the  agency  had 
planned  to  have  DCAA  evaluate  proposed  overhead  rates. 
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Effective  market  research  can  also  aid  in  determining 
price  realism.  By  identifying  adeguate  competition  through 
market  research  and  promulgating  the  solicitation  to 
numerous  qualified  and  capable  contractors,  the  FAR  has 
stated  that  such  competition  will  usually  establish  price 
reasonableness.  [Ref.  2:15.305] 

4 .  Equitably  and  Consistently  Evaluate  Past 
Performance 

Agencies  improperly  or  unfairly  evaluated  a 
contractor's  past  performance  in  20  percent  of  the 
sustained  protests  analyzed.  More  importantly,  it  is  one 
of  the  most  significant  evaluation  factors  because  it  is  a 
mandatory  consideration  on  all  procurements  over  $1 
million.  [Ref. 5:  p.594] 

U.S.  Property  is  a  classic  example  of  unfairly 
evaluating  a  contractor's  past  performance.  In  essence, 
two  newly  formed  companies — whose  presidents  had  both 
previously  worked  for  the  incumbent  contractor — each  claimed 
virtually  the  same  experience  in  their  respective 
proposals .  One  was  evaluated  as  having  acceptable  prior 
experience,  while  the  other's  experience  was  rated  as  only 
marginal.  As  in  the  last  section,  more  thorough  market 
research  may  have  eliminated  the  risk  of  a  sustained 
protest.  If  the  past  performance  stated  in  each  proposal 
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had  not  obviously  shown  that  both  companies'  senior 
leadership  shared  the  same  prior  work  experience,  then 
effective  market  research  might  have  brought  this 
information  into  plainer  view.  Comprehensive  discussions 
on  this  subject  should  have  made  the  inequity  apparent  to 
evaluators.  Instead  of  relying  on  evaluators  to  sift 
through  reams  of  proposal  paperwork  to  find  the  pertinent 
information,  adequately  structured  and  fairly  conducted 
discussions  could  have  assisted  in  leveling  the  playing 
field  as  far  as  past  performance  was  concerned. 

In  Nova,  the  company's  actual  past  performance  had 
been  rated  "average  to  outstanding"  by  previous  contracting 
agencies.  Evaluators  who,  instead,  focused  on  the  fact 
that  Nova  had  numerous  unresolved  contract  claims 
apparently  overlooked  this.  Had  these  same  evaluators 
delved  deeper  into  the  issue,  they  would  likely  have 
reached  the  same  conclusions  as  GAO.  After  examining  the 
claims  closely,  there  was  no  evidence  that  these  claims 
indicated  problems  in  management  effectiveness, 
responsiveness  or  cooperation.  [Ref. 13]. 

Finally,  personnel  must  be  instructed  on  how  to 
evaluate  a  lack  of  past  performance.  The  FAR  Part  15 
states  that,  in  the  case  of  an  offeror  without  a  record  of 
relevant  past  performance  or  when  past  performance 
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information  is  not  available,  the  offeror  may  not  be 
evaluated  favorably  or  unfavorably  on  past  performance 
[Ref.  2:15. 305(a) (2) (iv) ] 

This  presents  both  a  training  challenge  and  a 
management  concern.  The  challenge  is  to  teach  source 
selection  team  members  about  the  regulation,  as  well  as  how 
to  apply  the  regulation  when  evaluating  proposals.  The 
management  concern  is  attempting  to  ensure  that  an 
offeror's  lack  of  past  performance  information  does  not 
subliminally  taint  the  evaluator's  proposal  rating.  It  is 
one  thing  to  train  the  individual  regarding  the 
regulations,  but  it  is  another  matter  entirely  when  an 
individual's  perceptions  creep  into  the  evaluation  of  other 
factors,  such  as  technical  ability  to  perform  the  contract. 
Only  through  strict  adherence  to  the  specific  evaluation 
criteria  can  managers  ensure  that  source  selection 
personnel  have  not  inadvertently  prejudiced  a  prospective 
contractor. 

5.  Conduct  Meaningful  Discussions  that  Draw  Offerors 
into  Areas  of  their  Proposal  that  Require 
Amplification  or  Correction 

By  failing  to  conduct  meaningful  discussions  with 
offerors,  contracting  agencies  were  responsible  for  just 
over  20  percent  of  all  sustained  protests  in  calendar  years 
1998  and  1999.  The  purpose  of  discussions  is  to  point 
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offerors  toward  areas  of  their  proposals  that  are  unclear 
or  in  error,  and  to  then  allow  them  to  revise  their  offer. 
It  can  also  be  a  signal  that  the  solicitation  itself  did 
not  clearly  describe  the  agency's  requirement.  If  that  is 
the  case,  contracting  officers  should  cancel  the 
solicitation,  revise  it  and  then  resolicit  the  requirement, 
or  amend  the  current  solicitation  so  that  all  offerors  are 
proposing  based  on  the  same  requirements.  Aside  from  the 
requirement  to  hold  meaningful  discussions  with  offerors  in 
the  competitive  range,  there  is  also  a  requirement  to  give 
offerors  an  opportunity  to  address  any  adverse  past 
performance  information  on  which  they  have  not  previously 
had  an  opportunity  to  comment. 

Biospherics  goes  one  step  beyond  failing  to  conduct 
meaningful  discussions.  In  this  case,  the  agency  suggested 
that  the  contractor  review  its  proposal  for  any  possible 
further  savings  when,  in  fact,  the  agency  ultimately 
decided  the  offeror's  price  was  unrealistically  low.  Such 
misleading  communications  undermine  the  contractor's 
confidence  in  Government  as  a  fair  business  partner  and 
further  promote  the  "Us  versus  Them"  mentality  that 
contracting  agencies  must  constantly  battle.  This  is 
another  element  of  the  source  selection  process  that  can  be 
improved  trough  training.  Effective  training  can  not  only 
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increase  the  knowledge  level  of  source  selection  team 
members,  but  also  improve  their  interpersonal  communication 
skills.  Both  of  these  benefits  to  training  will  improve 
the  value  of  contract  discussions. 

Mechanical  is  a  protest  that  follows  the  reevaluation 
of  proposals  as  the  result  of  a  prior  sustained  protest 
involving  the  same  solicitation.  GAO  sustained  the  initial 
protest  noting  that  the  agency  had  not  adequately  supported 
it's  proposal  evaluations  and  because  the  evaluations 
lacked  reasonable  bases .  The  evaluation  board  in  the 
original  source  selection  had  evaluated  Mechanical  as 
excellent  or  better  in  three  technical  sub  factors  and  had 
therefore  not  held  discussions  regarding  those  sub  factors. 
In  the  follow  on  review,  the  evaluation  board  severely 
downgraded  the  offeror  in  those  same  sub  factors  but  did 
not  hold  discussions  regarding  the  newly  discovered 
deficiencies.  GAO  found  that  since  these  deficiencies  had 
not  been  addressed  in  previous  discussions  they  could  not 
form  the  basis  for  the  contracting  officer's  decision  to 
eliminate  the  offeror  from  consideration.  The  lesson  that 
contracting  agencies  should  derive  from  Mechanical  is  that 
discussions  must  be  tailored  to  the  specifics  of  the 
solicitation  and  the  unique  offer  in  order  to  be 
meaningful.  As  mentioned  previously,  meaningful 
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discussions  1)  should  lead  offerors  to  areas  of  their 
proposal  that  require  amplification  or  correction,  2)  must 
address  adverse  past  performance  information  which  the 
offeror  has  previously  not  had  the  opportunity  to  comment 
on,  and  3)  should  seek  to  level  the  playing  field  of  all 
offerors  without  resorting  to  technical  leveling  or 
technical  transfusion. 

6.  Craft  a  Complete  Solicitation  that  Addresses  the 
Agency's  True  Requirements  and  Clearly  Instructs 
Offerors  in  Proposal  Preparation  and  Evaluation 

Requirements  generation  and  acquisition  management 
represent  two  of  the  three  key  management  processes, 
discussed  in  DoD  Directive  5000.1,  15  March  1996,  that  must 
dovetail  effectively  to  ensure  success.  The  third  process 
is  the  planning,  programming  and  budgeting  system.  This 
philosophy  is  reflected  in  the  FAR  which  states  that  a 
request  for  proposal  must  contain  at  a  minimum  1) 
Government's  requirements,  2)  anticipated  terms  and 
conditions  that  will  apply  to  the  contract,  3)  information 
required  to  be  in  the  offers,  and  4)  factors  and  sub 
factors  that  will  be  used  to  evaluate  the  proposal  and 
their  relative  importance.  [Ref.  2:15. 203(a)] 
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Once  again  the  primary  deterrent  of  sustained  protests 
is  an  investment  of  time.  By  applying  significant  effort 
in  the  planning  stage  of  an  acquisition,  contracting 
agencies  can  help  ensure  that  contract  performance  is  on 
time,  within  budget  and  meets  the  needs  of  the  end  user. 

Contracting  agencies  are  afforded  broad  discretion  in 
the  evaluation  process.  Provided  that  proposals  are 
evaluated  reasonably  and  that  award  is  made  in  accordance 
with  the  evaluation  scheme  described  in  the  RFP,  GAO  will 
decide  for  the  Government. 

Another  planning  tool  contracting  agencies  can  use  in 
an  effort  to  reduce  the  risk  of  a  sustained  protest  is  to 
involve  prospective  contractors  early  in  the  process. 

Early  supplier  involvement  helps  ensure  that  the  user' s 
requirements  1)  are  technically  capable  of  being  met,  2) 
can  be  met  within  a  specified  timeframe,  and  3)  can  be 
accomplished  at  an  affordable  price.  Involving  industry  at 
the  start  of  the  acquisition  process  also  allows  for 
innovative  technologies  to  be  proposed  and  helps  foster  a 
sound  business  relationship  for  future  efforts. 

E .  CONCLUSION 

This  chapter  provided  a  qualitative  analysis  of  the  42 
cases  of  sustained  protests  of  contracts  awarded  using  a 
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trade-off  process.  First,  the  chapter  presented  statistics 
to  provide  a  better  understanding  of  the  types  and 
frequency  of  sustained  elements  observed  by  the  researcher. 

The  eight  sustaining  elements  observed  among  the  cases 
studied  were,  in  descending  order  of  frequency: 

•  Selection  Decision  not  Adequately  Documented 

•  Evaluation  of  Proposals  Inconsistent  with  RFP 

•  Improper  Agency  Evaluation  of  Cost/Price 

•  Improper  Agency  Evaluation  of  Past  Performance 

•  Agency  Improperly  Failed  to  Conduct  Meaningful 
Discussions 

•  Improper  Agency  Evaluation  of  Technical  Merit 

•  Improper  Agency  Evaluation  of  Labor 
Qualifications 

•  Failure  to  Adequately  Disclose  Factor  Evaluation 
Weights  in  Solicitation 

In  the  researcher's  opinion,  these  eight  common 
sustaining  elements  can  be  further  reduced  to  three  guiding 
principles  that  Federal  agency  contracting  officers  can 
follow  in  order  to  mitigate  the  risk  of  sustainable 
protests.  They  are: 

•  Treat  each  offeror  fairly,  ethically  and 
equitably,  both  in  relation  to  other  offerors  and 
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as  a  prospective  contractor  seeking  to  engage  in 
business  dealings  with  the  Government. 

•  Evaluate  each  and  every  proposal  strictly  in 
accordance  with  the  stated  evaluation  scheme  set 
forth  in  the  solicitation. 

•  Thoroughly  evaluate  the  full  merits  of  every 
proposal  that  is  submitted,  and  completely 
document,  with  reasoned  explanation,  every 
business  decision  made  on  behalf  of  the 
Government . 

Clearly  it  is  not  possible  to  completely  avoid 
protests  by  prospective  contractors,  but  through  reasoned 
and  even-handed  practices,  it  is  possible  to  mitigate  the 
risk  of  sustainable  protests. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

This  chapter  first  provides  answers  to  the  primary  and 
secondary  research  questions,  which  were  the  foundation  of 
this  thesis.  It  then  presents  conclusions  and 
recommendations  drawn  from  the  qualitative  analysis  in 
Chapter  IV.  Finally,  the  chapter  recommendsed  areas  for 
further  research  regarding  protests  of  contracts  awarded 
using  a  trade-off  process. 

B .  RESEARCH  QUESTIONS 

In  order  to  accomplish  the  objectives  of  this  thesis, 
fundamental  research  questions  were  formulated.  The 
responses  to  the  primary  and  secondary  research  questions 
are  provided  in  this  section. 

1.  Primary  Research  Question 

To  what  extent  will  an  analysis  of  General  Accounting 
Office  decisions  of  protests  of  contracts  awarded  using  the 
best  value  trade-off  process ,  provide  insight  into  the 
nature  of  Government  contract  protests  and  suggest  ways  to 
reduce  such  protests  in  the  future? 

Analyzing  sustained  protests  of  contracts  awarded 
using  a  best  value  trade-off  process  provided  a  significant 
understanding  of  the  basis  for  GAO's  decisions.  The 
following  are  selected  general  observations: 
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•  Protests  were  sustained  most  frequently  because 
of  an  agency's  failure  to  evaluate  proposals  in 
accordance  with  the  evaluation  scheme  and  because 
business  trade-offs  made  in  the  selection 
decision  were  not  adequately  supported  by 
reasoned  documentation. 

•  An  agency's  inability  to  properly  evaluate  all 
aspects  of  an  offeror's  proposal- cost /price,  past 
performance,  technical  merit  and  labor 

qualif ications-was  a  common  fatal  flaw  among 
sustained  protests. 

•  The  fact  that  a  Federal  agency  elects  to  use  a 
trade-off  source  selection  process  does  not 
increase  the  sustainment  rate  of  protests 
compared  to  all  protests  of  contracts  awarded. 

•  Many  protests  reflect  ineffective  communications 
between  offerors  and  contracting  agencies  during 
the  evaluation  process. 

The  following  are  suggestions  for  mitigating  the  risk 
of  sustained  protests  of  contracts  awarded  using  the  trade¬ 
off  process: 

•  Document  all  business  decisions  and  trade-offs 
made  in  evaluating  offerors’  proposals. 

•  Evaluate  proposals  strictly  in  accordance  with 
the  evaluation  scheme  stated  in  the 
solicitations . 

•  Conduct  thorough  and  adequate  evaluations  of  all 
aspects  of  an  offeror's  proposal. 

•  Conduct  meaningful  discussions. 

•  Structure  solicitations  to  adequately  reflect  the 
factor  evaluation  weights  to  be  applied  in 
proposal  evaluation. 
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2 .  Secondary  Research  Questions 

a.  What  will  an  analysis  of  Government  contract 
protests  from  January  1998  through  December 
1999  suggest  about  the  nature  of  Government 
contract  protests? 

Based  on  an  analysis  of  325  protests  of  contracts 
awarded  using  the  trade-off  process,  the  researcher 
observed  the  following  general  trends: 

•  Government  tends  to  win  seven  out  of  every 
eight  protests. 

•  The  number  of  protests  filed  against  DoD 
awards  was  nearly  two-fifths  greater  than 
for  all  other  Federal  agencies  combined. 

•  The  sustainment  rate  was  slightly  lower  for 
protests  of  DoD  awards  compared  to  all  other 
Federal  agencies  as  a  group. 

•  One  in  eight  protests  had  previously  been 
brought  before  the  Comptroller  General 
(i.e.,  same  contract  award). 

•  The  most  common  sustaining  element  was 
failure  to  adequately  document  the  source 
selection  decision. 

•  Defense  Finance  Accounting  Service  had  the 
highest  sustainment  rate  among  DoD 
activities,  with  67  percent. 

•  The  Department  of  Health  and  Human  Services 
had  a  sustainment  rate  of  40  percent,  the 
highest  among  other  Federal  agencies. 

•  DoD  accounted  for  exactly  half  of  the 
sustained  protests. 

b.  Do  sustained  protests  of  awards  using  the 
best  value  trade-off  process  fall  into 
distinct  categories? 
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The  results  of  the  researcher's  analysis  show 
that  sustained  protests  of  contracts  awarded  using  the 
trade-off  process  fall  into  one  or  more  of  the  following 
categories : 

•  Selection  Decision  Not  Adequately  Documented 

•  Evaluation  of  Proposals  Inconsistent  with  RFP 

•  Improper  Agency  Evaluation  of  Cost/Price 

•  Improper  Agency  Evaluation  of  Past  Performance 

•  Agency  Improperly  Failed  to  Conduct  Meaningful 
Discussions 

•  Improper  Agency  Evaluation  of  Technical  Merit 

•  Improper  Agency  Evaluation  of  Labor 
Qualifications 

•  Failure  to  Adequately  Disclose  Factor  Evaluation 
Weights  in  Solicitation 

c.  What  will  an  analysis  of  the  research  results 
suggest  about  weaknesses  in  Government 
contracting  norms  and  execution  practices? 

The  research  revealed  eight  categories  of 

sustaining  elements  in  which  the  Government  exhibited 

weaknesses  in  its  source  selection  norms  and  execution 

practices.  An  analysis  of  these  elements  indicates  that 

the  primary  weakness  is  the  failure  of  those  involved  in 

the  source  selection  process  to  adequately  support  the 

trade-off  decision  with  reasoned  documentation.  The  second 

most  frequent  weakness  observed  also  involved  various 
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members  of  the  source  selection  process.  Failure  to  adhere 

to  the  evaluation  scheme  set  forth  in  the  solicitation 

constituted  one-third  of  all  source  selection  weaknesses. 

The  research  indicated  that  there  is  a  need  for  on-going 

training  among  all  members  of  the  source  selection  team  and 

that  the  focus  of  such  training  should  be  in  the  proper 

methods  for  proposal  evaluation,  conduct  of  discussions  and 

source  selection  documentation. 

d.  How  can  the  results  of  this  analysis  be  used 
to  reduce  the  risk  of  protest? 

There  is  very  little  that  Federal  contracting 

agencies  can  do  to  reduce  the  risk  of  a  contract  protest. 

After  all,  prospective  contractors  have  little  to  lose;  for 

the  price  of  a  stamp,  they  can  file  a  protest  with  the 

Comptroller  General.  Frivolous  protests,  however,  will  be 

summarily  dismissed.  This  leaves  the  question:  "How  can 

the  number  of  legitimate  protests  be  reduced?"  The  answer 

can  be  found  in  the  following  two  steps: 

•  Ensure  the  integrity  of  the  source  selection 
process,  as  described  in  Chapter  II. 

•  Adequately  inform  unsuccessful  offerors,  during 
the  debriefing  process,  of  the  reasons  their 
proposal  was  not  selected. 

By  following  these  steps.  Federal  contracting 
agencies  can  reduce  the  risk  of  protest. 
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e.  How  can  the  results  of  this  analysis  be  used  to 
reduce  the  risk  of  sustainable  protest? 

This  research  revealed  that,  with  one  exception, 

the  basis  for  sustainment  occurred  during  the  evaluation  of 

proposals  and  resulted  in  the  Comptroller  General  ruling 

that  a  particular  aspect  of  the  source  selection  process 

was  unreasonable,  improper  or  unsupported.  In  order  to 

effectively  mitigate  the  risk  of  a  sustainable  protest, 

contracting  agencies  must  first  ensure  that  proposals  are 

properly  evaluated  in  accordance  with  governing  regulations 

and  the  specifics  of  the  solicitation.  Second,  contracting 

agencies  must  adequately  document  the  rationale  for  their 

trade-off  decisions.  Performing  this  second  step  not  only 

helps  protect  against  successful  protests  by  demonstrating 

that  the  decision  is  neither  arbitrary  nor  capricious,  but 

also  provides  a  framework  on  which  to  base  the  debriefing 

of  unsuccessful  offerors.  By  thoroughly  understanding  the 

reason  for  the  agency's  trade-off  decision,  unsuccessful 

offerors  begin  to  perceive  the  selection  process  as 

equitable  and  logical. 

C .  CONCLUSIONS 

The  answers  to  the  research  questions  have  led  the 
researcher  the  following  conclusions: 

•  Prospective  contractors’  protests  are  usually 
denied  unless  they  can  show  that  a  selection 
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decision  is  unreasonable  or  conflicts  with  the 
stated  evaluation  criteria  for  award. 

•  Contracting  agencies  would  benefit  from  training 
in  the  areas  of  source  selection  and  decision 
documentation . 

•  Communication  between  offerors  and  the  Government 
appears  to  be  problematic  with  respect  to 
discussions . 

•  Contracts  of  all  types  and  sizes  are  subject  to 
considerable  delay  due  to  protests.  This  was 
reflected  in  the  five  cases  that  had  been  brought 
before  the  Comptroller  General  a  second  time  and 
that  resulted  in  an  average  period  of  19  months 
from  release  of  RFP  until  ultimate  sustainment  by 
GAO.  This  does  not  even  indicate  the  extent  of 
the  delay  until  actual  contract  performance 
begins . 

D .  RECOMMENDATIONS 

This  research  suggests  the  following  recommendations 
for  mitigating  the  risk  of  sustainable  protests: 

•  Adequately  support  the  selection  decision  with 
reasoned  documentation. 

•  Evaluate  proposals  in  a  manner  consistent  with  the 
solicitation. 

•  Thoroughly  conduct  cost/price  realism  analyses  when 
required,  and  always  evaluate  the  total  cost/price. 

•  Equitably  and  consistently  evaluate  past  performance. 

•  Conduct  meaningful  discussions  that  draw  offerors  into 
areas  of  their  proposal  that  require  amplification  or 
correction. 

E.  AREAS  FOR  FURTHER  RESEARCH 

The  scope  of  this  research  was  limited  to  protests  of 
contracts  awarded  using  the  trade-off  process  for  calendar 
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years  1998  and  1999,  with  attention  focused  on  protests 
that  were  sustained  and  the  reasons  for  sustainment. 
Specific  areas  that  merit  further  research  include: 

•  Comparison  of  GAO  decisions  with  those  of  the  U.S. 
Court  of  Federal  Claims. 

•  Analysis  of  U.S.  Court  of  Federal  Claims  decisions  for 
a  similar  period. 

•  Analysis  of  those  contracting  agencies  with  the 
highest  protest  sustainment  rates,  focusing  on  why 
these  agencies  have  higher  sustainment  rates  than 
other  agencies. 

•  Continuation  of  this  thesis,  focusing  on  types  of 
products  or  services  being  solicited. 

•  Development  of  a  center  of  excellence  training  program 
for  training  source  selection  teams  that  will  be 
awarding  contracts  using  the  trade-off  process. 

•  A  study  determining  if  early  supplier  involvement  and 
teaming  with  industry  practices  have  affected  the 
level  of  protests  of  contracts  awarded  using  the 
trade-off  process. 
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APPENDIX  A.  SELECTED  ACRONYMS 


Army 

BAFO 

CO 

COFD 

CPAF 

CPFF 

DCA 

DFARS 

DFAS 

DISA 

DLA 

DoA 

DoC 

DoD 

DoE 

DoJ 

DoN 


CoE  Army,  Corps  of  Engineers 

Best  and  Final  Offer  (see  FPR) 

Contracting  Officer 

Contracting  Officer's  Final  Decision 

Cost-Plus-Award-Fee 

Cost-Plus-Fixed-Fee 

Defense  Commissary  Agency 

Defense  Federal  Acquisition  Regulation 

Supplement 

Defense  Finance  and  Accounting  Service 

Defense  Information  Systems  Agency 

Defense  Logistics  Agency 

Department  of  Agriculture 

Department  of  Commerce 

Department  of  Defense 

Department  of  Energy 

Department  of  Justice 

Department  of  Navy 
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DoT 

DSCP 

DSCR 

EPA 

FAR 

FFP 

FHA 

FPAF 

FPIF 

GAO 

GSA 

HHS 

HUD 

IDIQ 

JFK-CPA 

LPTA 


Department  of  Transportation 
Defense  Supply  Center  Philidelphia 
Defense  Supply  Center  Richmond 

Environmental  Protection  Agency 

Federal  Acquisition  Regulation 
Firm  Fixed-Price 
Federal  Highway  Administration 
Fixed-Price  Award  Fee 
Fixed-Price- Incentive-Firm 

General  Accounting  Office 
General  Services  Administration 

Health  &  Human  Services  (Department  of) 
Housing  &  Urban  Development  (Department  of) 

Indefinite  Delivery,  Indefinite  Quantity 

J.F.  Kennedy,  Center  for  the  Performing  Arts 

Lowest  Priced,  Technically  Acceptable 
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NASA 

NIEHS 

PCC 

t 

RFP 

RFQ 

SSA 

SSAC 

SSEB 

SSP 

SS  Admin 

TSO 

USAID 

USMS 

USGS 

VA 


National  Aeronautical  &  Space  Administration 
National  Institute  of  Environmental  Health 
Service 

Panama  Canal  Commission 

Request  for  Proposal 
Request  for  Quotation 

Source  Selection  Authority 
Source  Selection  Advisory  Council 
Source  Selection  Evaluation  Board 
Source  Selection  Plan 
Social  Security  Administration 

TRICARE  Support  Office 

U.S.  Agency  for  International  Development 
U.S.  Marshal  Service 
U.S.  Geological  Survey 

Veteran's  Affairs  (Department  of) 
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APPENDIX  B.  AMC  PAMPHLET  715-3 


AMC  Pamphlet  715-3 

Contracting  for  Best  Value 
A  Best  Practices  Guide  to  Source  Selection 

1  January  1998 


Department  of  the  Army 

Headquarters,  United  States  Army  Materiel  Command 
5001  Eisenhower  Avenue,  Alexandria,  VA  22333-0001 

Foreword 


This  guide  provides  techniques  and  practices  for  obtaining  best  value  products  and  services 
through  source  selection.  Consistent  with  the  spirit  of  acquisition  reform,  it  introduces  new  and 
innovative  techniques  to  simplify  the  source  selection  process  and  produce  better  value.  Its 
purpose  is  to  provide  you  with  a  practical  reference  tool  that  will  help  you  implement  a  new  way 
of  doing  business  that  promotes  flexibility,  streamlining,  and  simplified  procedures. 

This  guide  is  designed  for  use  by  the  entire  acquisition  workforce  to  promote  a  consistent 
understanding  of  best  value  and  the  various  processes  and  techniques  that  can  be  used  to  achieve 
it.  It  explains  best  practices  for  planning  your  source  selection,  teaming,  exchanging  information 
with  industry,  and  conducting  efficient  and  effective  source  selections. 

I  encourage  you  to  read  and  use  this  guide  in  your  efforts  to  get  the  best  value  for  your 
customers. 


Johnnie  E.  Wilson 
General,  USA 
Commanding 

U.S.  Army  Materiel  Command 
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and  effective  source  selections  that  produce  better  value.  It  should  be  used  in  conjunction  with 
existing  regulations. 
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113 


contributions.  We  also  extend  our  thanks  to  Curtis  Stevenson  from  the  Army  staff  for  his 
support  and  advice.  Finally,  we  acknowledge  and  thank  all  of  the  Source  Selection  Improvement 
Integrated  Product  Team  members  and  the  individuals  within  their  networks  across  the  Army 
who  reviewed  the  drafts,  contributed  invaluable  comments  and  recommendations,  and  so 
graciously  shared  their  knowledge  and  experience. 

We  encourage  you  to  share  this  guidance  with  those  who  may  benefit. 

We  are  distributing  only  a  limited  number  of  hard  copies  since  we  have  made  the  guide 
electronically  accessible  as  follows: 

You  may  view,  download  and  provide  feedback  on  this  guide  via  the  Internet  through  the  AMC 
Source  Selection  Resource  Center  at  http://www.army-acquisition.net/amcweb/fr_ssll.htm. 

This  guide  is  also  included  within  the  DOD  Acquisition  Deskbook,  accessible  via  the  Internet  at 
http://web.deskbook.osd.mil  or  on  CD-ROM. 

The  Source  Selection  “©  Blueprints”  referenced  throughout  the  guide  are  designed  to  be  living 
models  for  commonly  used  source  selection  documents.  While  they  are  not  available  with  this 
hard  copy,  they  may  be  accessed  electronically*. 


This  pamphlet  does  not  create  any  substantive  or  procedural  right  in 
third  parties  or  impose  any  specific  legal  duty  or  obligation  upon  any 
government  organization  or  employee. 


This  guide  consolidates  and  updates  all  AMC  source  selection  guides  into  a  single  document  that 
includes  separate  appendices  on  past  performance,  oral  presentations,  and  debriefings.  It 
should  be  used  to  promote  best  value  in  source  selection. 

*  This  pamphlet  supersedes  AMC-P  715-3  VoL  1,  8  Jan  87;  AMC-P  715-3  Vol. 

2,  9  Oct  87;  AMC-P  715-3  Vol  3, 18  Mar  87;  AMC-P  715-3  Vol  4, 15  Sep 
93;  AMC-P  715-3  Vol  5, 16  Aug  94;  and  AMC-P  715-3  Vol  6,  24  Jan  95. 
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Overview 


Source  selection  is  the  process  used  in  competitive,  negotiated  contracting  to  select  the  proposal 
expected  to  result  in  the  best  value  to  the  Government.  The  source  selection  approach  must  be 
tailored  to  the  acquisition.  You  have  to  consider  your  evaluation  needs;  i.e.,  don’t  make  source 
selection  more  complicated  and  expensive  than  necessary. 

What  is  Best  Value? 

In  the  broadest  sense,  best  value  is  the  outcome  of  any  acquisition  that  ensures  we  meet  the 
customer’s  needs  in  the  most  effective,  economical,  and  timely  manner.  It’s  the  result  of  the 
unique  circumstances  of  each  acquisition,  the  acquisition  strategy,  choice  of  contracting  method, 
and  award  decision.  Under  this  concept,  best  value  is  the  goal  of  sealed  bidding,  simplified 
acquisition,  commercial  item  acquisition,  negotiated  acquisition,  and  any  other  specialized 
acquisition  methods  or  combination  of  methods  you  choose  to  use. 


Best  Value  is  the  goal 
of  every  acquisition 


Purpose  of  This  Guide 

This  guide  provides  information  on  the  various  processes  and  techniques  that  can  be  used  to 
conduct  efficient  and  effective  source  selections.  We  are  presenting  some  of  the  best  and  most 
innovative  practices  being  used.  The  principles  in  this  guide  apply  to  all  source  selections,  both 
those  that  are  complex  and  the  majority  where  the  contracting  officer  is  the  selection  official. 
During  acquisition  planning,  select  the  methodology  that  is  most  appropriate  to  the  unique 
circumstances  of  the  acquisition  and  expected  to  result  in  the  best  value. 

Unless  you  use  a  lowest  price  technically  acceptable  evaluation  approach,  your  source  selection 
will  involve  some  form  of  tradeoff.  This  guide’s  focus  is  on  the  tradeoff  process  and  will 
provide  some  hints  and  ideas  that  will  be  useful  in  doing  a  tradeoff  between  cost  or  price  and 
other  important  factors. 

There  are  two  important  points  to  keep  in  mind  as  you  do  your  planning  and  select  your 
evaluation  and  source  selection  process: 

•  Tailor  your  process  to  fit  your  circumstances.  There  is  no  magic  checklist  in  this 
arena.  Consider  the  complexity  of  the  acquisition  and  resources  available.  Use  a 
combination  of  techniques  if  it  will  work  best  for  you  and  if  it  is  fair. 

•  The  same  principles  apply  in  selecting  and  executing  a  source  selection  process  or 
technique,  whether  you  are  using  a  formally  structured  organization  for  a  complex 
acquisition  or  a  more  streamlined  process  typical  for  the  majority  of  source 
selections. 
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The  Federal  Acquisition  Regulation  (FAR)  prescribes  the  general  policies  governing  source 
selection. 

Appendix  A  of  this  guide  contains  definitions  of  certain  words  and  terms  associated  with  source 
selection. 

Appendix  B  contains  references  along  with  their  Internet  addresses  to  assist  you  further  in 
contracting  for  best  value  and  conducting  a  source  selection. 

The  Source  Selection  “©Blueprints”  referenced  throughout  this  guide  are  models  for  commonly 
used  source  selection  documents  and  procedures  and  should  be  tailored  to  each  individual 
acquisition.  They  may  be  accessed  only  on  electronic  versions,  e.g.,  http://www.army- 
acquisition.net/amcweb/fr_ssll.htm. 

Importance  of  the  Source  Selection  Authority 

The  consequences  of  the  selection  decision  can  be  far-reaching.  In  most  cases  the  contracting 
officer  is  the  selection  official.  In  some  acquisitions,  or  class  of  acquisitions,  the  agency  head  or 
other  official  may  be  the  selection  official,  or  will  appoint  someone  else  to  make  the  selection. 
The  source  selection  authority  must  be  at  a  level  that  is  fully  accountable  for  the  results  of  the 
decision  and  knowledgeable  of  the  factors  necessary  to  determine  the  best  value.  In  addition, 
successful  execution  of  an  acquisition  using  the  tradeoff  process  requires  early  involvement  of 
the  source  selection  authority  so  that  person  is  prepared  to  make  a  rational  selection  decision 
consistent  with  the  solicitation.  The  amount  of  time  and  effort  required  obviously  needs  to  be 
considered  when  making  the  appointment. 

In  a  complex  source  selection,  it  may  be  useful  to  provide  a  number  of  briefings  to  the  source 
selection  authority  early  in  the  acquisition  process  and  at  critical  steps  throughout  the  process. 
This  approach  will  ensure  that  the  source  selection  authority  knows  the  program  and  the 
acquisition  process  constraints.  It  also  allows  the  source  selection  authority  to  readily  express 
concerns  and  ideas  that  are  likely  to  influence  the  final  selection  decision. 

Examples  of  where  source  selection  authority  involvement  is  essential  include  approval  of  the 
source  selection/evaluation  plan  and  the  solicitation. 

Importance  of  Procurement  Integrity 

There  are  stringent  requirements  for  maintaining  the  integrity  of  the  procurement  process  that 
Must  be  adhered  to  by  all  participants  involved  in  the  source  selection  process.  This  includes 
both  technical  and  contracting  personnel.  Procurement  integrity  rules  provide  for  both  civil  and 
criminal  penalties  for  violations  (see  FAR  3.104).  The  guiding  principle  behind  these 
requirements  is  that  all  offerors  are  treated  fairly  and  no  one  obtains  an  unfair  advantage. 


116 


Planning  For  Source  Selection 

Designing  an  Acquisition  Strategy 

As  soon  as  possible  after  a  need  to  acquire  products  or  services  has  been  identified,  an 
acquisition  strategy  meeting  should  be  held.  The  attendees  should  include  the  person  responsible 
for  managing  the  program  or  project,  acquisition  and  legal  representatives,  potential  evaluation 
team  members,  and  others  as  needed. 

The  strategy  meeting  should  be  used  to  determine  the  acquisition  approach  including  the  source 
selection  process  and  techniques  that  will  be  most  appropriate.  The  group  should  use  the 
meeting  to  discuss  the  results  of  market  research,  potential  evaluation  factors,  information  that 
may  be  needed  from  offerors  to  support  those  factors,  and  other  appropriate  planning  issues  such 
as  the  timetables  for  the  acquisition  and  who  should  be  members  of  the  evaluation  team.  The 
group  should  design  a  strategy  that  best  reflects  the  specific  requirement,  the  results  of  market 
research,  and  the  risks  associated  with  the  acquisition.  The  information  obtained  in  the  strategy 
meeting  will  be  used  as  a  basis  for  developing  the  source  selection/evaluation  plan. 


Source  selection 
is  a  team  effort 


Forming  a  Team 
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Source  Selection  “Blueprint” 
Typical  Complex  Source  Selection  Team 


Blueprint:  Typical  Complex  Team 
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Source  Selection  “Blueprint” 

Typical  Non-Complex  Source  Selection  Team 


SOURCE  SELECTION  AUTHORITY  (SSA) 
(usually  the  contracting  officer) 
COMPARES  AND  DECIDES 


EVALUATOR 
Evaluates  Cost 


EVALUATOR 
Evaluates  Technical  Merit 
and  Proposal  Risk 


EVALUATOR 
Evaluates 
Performance  Risk 


Evaluators  evaluate  proposals  against 
RFP  factors  and  subfactors.  Assist  the 
SSA  in  comparative  analysis,  if  requested 


Blueprint:  Typical  Non-Complex  Team 

Source  selection  should  be  a  multidisciplined  team  effort  from  the  earliest  planning  stages. 

The  size  and  composition  of  the  team  should  be  tailored  specifically  to  the  acquisition.  In 
complex  source  selections  you  may  have  a  larger  team  (e.g.,  8  to  10  people)  from  various 
functional  disciplines.  In  streamlined  source  selections,  however,  the  team  may  consist  of  one  or 
more  technical  evaluators  and  the  contracting  officer,  who  is  also  the  source  selection  authority. 
Whether  the  team  is  large  or  small,  it  should  be  established  to  ensure  continuity  and  active 
ongoing  involvement  of  appropriate  contracting,  technical,  logistics,  legal,  user,  contract 
administrators,  and  other  experts  to  ensure  a  comprehensive  evaluation  of  each  proposal. 

Researching  the  Market 

Market  research  is  the  first  step  in  any  acquisition  and  an  essential  part  of  designing  every 
acquisition  strategy.  The  acquisition  team  uses  market  research  to  obtain  information  on 
products  and  services  available  in  the  commercial  marketplace.  Market  research  is  key  in 
determining  whether  a  need  can  be  met  by  a  commercial  item  or  nondevelopmental  item  and  in 
identifying  commercial  practices  associated  with  such  items  or  services.  It  also  has  a  key  impact 
on  your  choice  of  appropriate  evaluation  factors,  contracting  method,  and  the  amount  and  type  of 
information  to  be  included  in  proposals. 
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A  thorough  research  of  the  market  should  be  done  as  soon  as  needs  are  forecast  and  as  part  of 
acquisition  planning.  Sometimes  it  might  be  a  one-person  effort.  Other  times  a  team  effort.  A 
variety  of  techniques  may  be  used  to  conduct  market  research  and  may  include: 

•  Contacting  knowledgeable  individuals  regarding  market  capabilities; 

•  Reviewing  the  results  of  recent  market  research; 

•  Querying  government  or  commercial  data  bases; 

•  Participating  in  interactive,  on-line  communication; 

•  Reviewing  catalogs  and  product  literature. 

Determining  the  Source  Selection  Approach 

One  of  the  first  steps  in  designing  an  acquisition  strategy  is  to  determine  the  source  selection 
approach  or  combination  of  approaches  that  you  will  use  to  obtain  the  best  value.  At  either  end 
of  the  best  value  continuum,  are  the  tradeoff  process  and  the  lowest  price  technically  acceptable 
process. 

Other  source  selection  processes  can  be  designed  to  fit  particular  circumstances.  You  could 
tailor  the  process  to  combine  elements  of  these  two  approaches.  You  could  also  use  oral 
presentations  as  part  of  the  proposal  submission.  The  point  is  that  the  source  selection  processes 
or  techniques  must  be  appropriate  to  the  acquisition. 

The  Tradeoff  Process 

Cost  or  price  is  always  an  evaluation  factor  in  any  source  selection.  However,  many  times  you 
may  have  other  factors  that  you  also  want  to  consider.  You  may  need  technical  capabilities, 
qualifications,  or  experience  that  a  low  cost/price  offeror  may  not  possess.  These  factors  may  or 
may  not  be  more  important  than  cost/price,  but  they  do  have  a  strong  bearing  on  the  source 
selection  decision.  The  source  selection  authority  needs  flexibility  to  select  the  best  value  that 
may  not  be  the  lowest  price  or  the  highest  technically  rated  offeror.  The  decision  will  involve  a 
comparison  of  the  combination  of  noncost  strengths,  weaknesses,  and  risks  and  cost/price  offered 
in  each  proposal  and  judgment  as  to  which  provides  the  best  combination.  The  source  selection 
authority  will  have  to  document  the  decision  and  why  the  selected  source  represents  the  best 
value  to  the  government.  This  is  the  essence  of  the  tradeoff  process. 

When  to  Use  the  Tradeoff  Process:  Strengths  and  Potential  Pitfalls 

Use  the  tradeoff  process  when  it  is  essential  to  evaluate  and  compare 
factors  in  addition  to  cost  or  price  in  order  to  select  the  most 
advantageous  proposal  and  obtain  the  best  value. 

The  tradeoff  process  is  particularly  appropriate  if: 

•  The  Government’s  requirements  are  difficult  to  define,  complex,  or  historically 
troublesome; 
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•  You  expect  measurable  differences  in  the  design,  performance,  quality,  reliability,  or 
supportability; 

•  Services  are  not  clearly  defined  or  highly  skilled  personnel  are  required; 

•  You  are  willing  to  pay  extra  for  capability,  skills,  reduced  risk,  or  other  noncost 
factors,  if  the  added  benefits  are  worth  the  premium; 

Always  consider  the  strengths  and  potential  pitfalls  of  using  a  tradeoff  process  to  ensure  that  it  is 

consistent  with  your  overall  acquisition  strategy. 

Strengths 

•  Allows  greater  flexibility  to  subjectively  compare  technical  and  cost  factors  to 
determine  the  value  of  the  relative  strengths,  weaknesses,  and  risks  of  the  proposals. 

•  Enables  selection  of  the  best  approach  among  a  range  of  solutions  and  increases  the 
likelihood  of  selecting  suppliers  who  are  most  likely  to  provide  quality  products  and 
services,  on  time,  and  at  reasonable  cost/price. 

•  Takes  advantage  of  the  experience  and  independent  judgment  of  the  source  selection 
official. 

Potential  Pitfalls 

•  Using  evaluation  factors  and  subfactors  that  are  not  derived  from  the  market  place 
and  do  not  accurately  reflect  the  Government’s  requirements.  This  may  result  in 
award  to  an  offeror  that  may  not  be  the  best  value. 

•  Using  too  many  evaluation  factors  and  subfactors.  A  large  number  of  factors  and 
subfactors  dilutes  consideration  of  those  which  are  truly  important. 

•  Failure  to  make  the  appropriate  investment  in  resources  needed  for  a  competent  and 
defensible  value  analysis. 

•  An  inherently  subjective  process,  and  thus  more  difficult  to  evaluate  and  document. 

Major  Steps  in  the  Tradeoff  Process 

The  tradeoff  process  generally  consists  of  the  following  steps: 

•  Designing  a  strategy  that  best  reflects  the  results  of  market  research  and  the  specific 
circumstances  of  the  acquisition. 

•  Establishing  and  documenting  a  source  selection  or  technical  evaluation  plan.  This 
plan  includes  the  acquisition  goals  and  objectives,  identification  and  relative 
importance  of  evaluation  factors  and  subfactors,  the  evaluation  standards,  and  the 
selection  process. 
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•  Structuring  the  solicitation  to  effectively  communicate  the  Government’s 
requirements,  mission  objectives,  the  factors  and  subfactors,  their  relative 
importance,  the  information  offerors  must  submit  for  evaluation  against  the  stated 
factors  and  subfactors  and  the  methodology  for  evaluating  the  proposals. 

•  Evaluating  the  offers  on  the  basis  of  the  source  selection  plan  and  the  evaluation 
factors  and  subfactors  in  the  solicitation  and  having  discussions  as  needed. 

•  Comparing  the  strengths,  weaknesses,  risks,  and  cost/price  or  most  probable  costs  of 
the  proposals  and  deciding  which  combination,  in  accordance  with  the  solicitation 
factors  and  subfactors,  represents  the  best  value. 

•  Documenting  the  source  selection  decision  including  the  tradeoffs  and  rationale 
used. 

•  Awarding  the  contract,  notifying  offerors  and  debriefing  them  upon  their  request. 

•  Documenting  the  lessons  learned  that  may  benefit  future  source  selections. 

The  Lowest  Price  Technically  Acceptable  Process 

In  some  situations,  simply  comparing  the  cost  or  price  of  proposals  meeting  or  exceeding  the 
solicitation’s  requirements  for  acceptability  can  be  expected  to  result  in  the  best  value.  In  such 
cases,  cost/price  is  the  overriding  consideration.  While  there  may  be  a  need  for  discussions  there 
is  no  need  to  make  tradeoffs. 

The  lowest  price  technically  acceptable  process  is  similar  to  a  sealed  bid  approach  in  that  award 
is  made  to  the  acceptable  offeror  with  the  lowest  evaluated  cost  or  price.  The  major  difference  is 
that  discussions  can  be  held  with  offerors  prior  to  source  selection  to  ensure  offerors  understand 
the  requirements  and  to  determine  acceptability.  Tradeoffs  are  not  permitted  and  no  additional 
credit  is  given  for  exceeding  acceptability.  However,  proposals  are  evaluated  to  determine 
whether  they  meet  the  acceptability  levels  established  in  the  solicitation  for  each  noncost 
evaluation  factor  and  subfactor. 

The  lowest  price  technically  acceptable  process  may  be  appropriate  where  the  requirement  is  not 
complex  and  the  technical  and  performance  risks  are  minimal,  such  as  acquisitions  where 
service,  supply,  or  equipment  requirements  are  well  defined  but  where  discussions  may  be 
necessary. 

Major  Steps  in  the  Lowest  Price  Technically  Acceptable  Process 

The  lowest  price  technically  acceptable  process  generally  consists  of  the  following  steps: 

•  Designing  a  strategy  that  best  reflects  the  results  of  market  research  and  the  specific 
circumstances  of  the  acquisition. 

•  Establishing  and  documenting  a  source  selection  or  technical  evaluation  plan.  This 
plan  includes  the  acquisition  goals  and  objectives,  identification  of  acceptability 
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requirements  for  each  noncost  evaluation  factor  and  subfactor,  and  procedures  for 
evaluating  proposals  and  making  award. 

•  Structuring  the  solicitation  to  effectively  communicate  the  Government’s 
requirements,  the  factors  and  subfactors  with  associated  acceptability  standards,  the 
information  offerors  must  submit  for  evaluation  of  acceptability  against  the  stated 
factors  and  subfactors,  and  the  basis  for  award  (i.e.,  the  lowest  priced  proposal 
meeting  or  exceeding  the  standards.) 

•  Evaluating  and  rating  proposals  on  a  pass/fail  basis  against  the  acceptability 
requirements  in  the  solicitation. 

•  Conducting  discussions  or  other  exchanges  as  needed.  Comparing  the  cost  or  prices 
of  acceptable  proposals  and  awarding  the  contract  to  the  offeror  with  the  lowest 
evaluated  price  meeting  the  acceptability  requirements. 

Past  Performance  and  the  Lowest  Price  Technically  Acceptable  Process 

•  If  you  determine  that  past  performance  is  a  discriminator  under  this  approach,  then 
you  must  state  in  the  solicitation  the  criteria  that  you  will  use  to  evaluate  it  on  a 
pass/fail  basis.  For  small  businesses,  an  unacceptable  rating  in  this  area  is  a  matter 
of  responsibility.  Therefore,  in  your  acquisition  planning,  you  should  anticipate  a 
possible  need  to  obtain  a  Certificate  of  Competency  from  the  Small  Business 
Administration  if  a  small  business  otherwise  eligible  for  award  has  unacceptable  past 
performance. 

•  A  Certificate  of  Competency  determination  is  not  required  however,  if  you  select  a 
hybrid  strategy  that  combines  the  lowest  price  technically  acceptable  and  tradeoff 
processes.  Under  such  a  strategy,  you  could  still  evaluate  technical  proposals  on  a 
pass/fail  basis  while  basing  the  final  selection  decision  on  a  tradeoff  between  past 
performance  and  price. 

Conducting  a  Presolicitation  Dialogue  with  Industry 
Foster  a  presolicitation  dialogue  with  industry  to: 

•  Ensure  a  mutual  understanding  of  the  government’s  need  and  industry’s 
capabilities 

•  Minimize  inclusion  of  non-value  added  requirements ,  and 

•  Promote  a  more  effective  source  selection. 

•  An  effective  dialogue  with  industry  even  before  a  solicitation  is  written  or  released 
can  pay  dividends  during  later  phases  of  the  process.  The  earlier  and  more  effective 
you  are  in  keeping  up  to  date  on  the  market  and  new  technology,  and  ensuring  the 
market  knows  what  your  requirements  are,  the  better  for  both  parties.  The  growing 
trend  is  to  provide  more  information,  not  less,  to  potential  offerors.  With  more 
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information,  they  can  make  informed  decisions  about  whether  to  compete,  they  can 
offer  better  proposals,  the  evaluation  and  selection  process  will  be  quicker  and 
smoother,  and  there  is  less  chance  of  miscommunication  and  a  protest.  There  are  a 
variety  of  mechanisms  to  maintain  contact  with  potential  offerors  including  the 
following: 

Advanced  Planning  Briefings  for  Industry  to  provide  a  forecast  of  future  direction 
and  requirements; 

Market  research  to  stay  abreast  of  innovation,  advances,  and  capabilities; 

Information  centers  to  provide  access  information  to  documents  relevant  to  the 
acquisition; 

Requests  for  Information  and  Draft  Requests  for  Proposals  to  obtain  information 
from  industry  on  such  things  as  price  and  availability  and  comments  on  the  proposed 
solicitation; 

Meetings  and  conferences,  including  one-on-one  meetings  with  potential  offerors 
and  Presolicitation  Conferences. 

In  conducting  a  presolicitation  dialogue  with  industry,  always  make  sure  that  you: 

Release  information  to  all  potential  offerors  on  a  fair  and  equitable  basis  consistent 
with  regulatory  and  legal  restrictions. 

Establish  clear  ground  rules  for  the  conduct  timing,  and  documentation  of  any  one- 
on-one  meetings  to  ensure  potential  offerors  are  given  equal  access  to  information 
needed  to  prepare  proposals. 

Protect  any  proprietary  information  that  you  are  given  access  to  during  this  process. 

Request  contracting  and  legal  counsel  advice  if  any  questions  arise  about 
presolicitation  exchanges. 


Drafting  a  Source  Selection/Evaluation  Plan 


A  thoroughly  contemplated  plan  for  selecting  a  best  value  source  is  vital  to  any  source  selection 
process.  In  all  source  selections,  the  plan  is  tailored  to  reflect  the  complexity  of  the  acquisition. 
In  more  complex  source  selections,  this  plan  is  called  the  Source  Selection  Plan  and  should  be 
prepared  for  the  source  selection  authority’s  approval.  In  less  complex  acquisitions  the  plan  is 
often  referred  to  as  the  Technical  Evaluation  Plan.  The  plan  is  developed  prior  to  or 
concurrently  with  preparation  of  the  solicitation.  It  states  your  intentions  for  organizing  and 
conducting  the  evaluation  and  analysis  of  proposals  and  the  source  selection.  It  contains 
acquisition  sensitive  information  and  is  not  released  outside  the  contracting  activity’s  source 
selection  organization. 
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Acquisitions  using  a  tradeoff  process  are  often  subject  to  dynamic  internal  and  external 
influences.  Examples  of  such  influences  include: 

•  The  differing  missions  or  functions  to  be  supported.  Such  situations  influence  how 
the  agency  specifies  its  requirements,  which  in  turn  influence  offerors’  solutions. 

•  The  rate  at  which  technology  and  market  factors  are  changing.  Between  the  time  the 
agency  identifies  a  requirement  and  the  offerors  submit  proposals,  technology  may 
have  developed  efficiency  and  productivity  benefits  unanticipated  by  the  agency. 
Accordingly,  you  should  structure  the  selection  plan  and  the  solicitation  to  consider 
these  influences  and  assure  that  the  proposal  selected  provides  the  best  value  to  the 
government. 

Purpose  of  the  Source  Selection  Plan 

The  source  selection  plan  serves  several  purposes,  including  — 

•  Defining  a  specific  approach  for  soliciting  and  evaluating  proposals. 

•  Describing  the  evaluation  factors  and  subfactors,  their  relative  importance,  and  the 
methodology  used  to  evaluate  proposals. 

•  Providing  essential  guidance  to  the  solicitation  developers,  especially  for  putting 
together  the  solicitation  sections  dealing  with  proposal  preparation  and  evaluation. 

•  Serving  as  a  charter  and  guide  for  the  source  selection  team  on  the  roles  of  the 
members  and  the  conduct  of  the  entire  source  selection  from  proposal  evaluation, 
through  the  cost/price/technical  tradeoff,  award  decision,  and  debriefing. 

Guidelines  for  a  Source  Selection  Plan 


Source  Selection/Evaluation  Plan 
Section  I.  Summary  Description 

A.  Item/Service  to  be  Acquired 

B.  Proposed  Contract  Type  (Choose  one) 

_ FFP 

_ FP  with  Incentives 

_ Cost 

_ Cost  with  Incentives 

_ Hybrid  (Explain) 

C.  Source  Selection  Process  ( Choose  one) 

Tradeoff 
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_ Low  Price  Technically  Acceptable 

_ Hybrid  (explain) 

D.  Special  Evaluation  Techniques  (explain,  if  applicable) 
e.g.  oral  presentations,  multi-step  techniques 

Section  II.  Source  Selection  Team 

A.  Members.  List  the  evaluation  team  members  by  name  and  functional  area  (including 
advisors,  if  applicable). 

B.  Team  Member  Certifications:  Verify/attach  by  reference  certificates  from  each  team 
member  protecting  unauthorized  release  of  source  selection  or  proprietary  information. 

Section  III.  Evaluation  Factors  and  Snbfactors 

A.  Proposal  Evaluation  Information.  State  all  factors,  subfactors  and  their  relative  order  of 
importance  exactly  as  they  will  appear  in  the  solicitation.  If  you  elect  to  include  desirable 
objectives  or  features  you  would  be  willing  to  pay  extra  for,  include  an  explanation  of  how  they 
will  be  evaluated  and  whether  or  not  credit  will  be  given  for  exceeding  such  desirables. 

B.  Proposal  Submission  Information.  State  the  instructions  to  offerors  for  preparing  and 
submitting  proposals  exactly  as  they  will  appear  in  the  solicitation. 

Section  IV.  Description  of  Evaluation  Procedures 

A.  Rating  System.  Describe  the  evaluation  technique  (ratings  and  their  definitions)  that  will  be 
used  to  rate  the  proposals. 

B.  Evaluation  Standards.  Describe  the  standards  or  target  levels  you  will  use  to  measure  how 
well  a  proposal  addresses  each  evaluation  factor  and  subfactor. 

C.  Procedures  for  Recording  the  Evaluation 

e.g.  Worksheet(s)  for  recording  evaluator  ratings  with  supporting  narrative 

D.  Schedule.  Identify  the  schedule  for  key  source  selection  activities. 

Section  V.  Rationale  for  the  Evaluation  Scheme 

A.  Factors  and  Subfactors.  Explain  how  the  selected  factors  and  subfactors  reflect  the 
circumstances  of  your  particular  acquisition  and  the  results  of  market  research  and  other 
presolicitation  exchanges  with  industry. 


B.  Weights.  Explain  how  the  relative  importance  of  the  factors  and  subfactors  reflect  the 
acquisition’s  primary  objective.  Explain  why  the  most  heavily  weighted  factors  are  the  “drivers’ 
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of  the  source  selection. 


Blueprint:  Typical  Evaluation  Plan 

Although  there  isn’t  a  specific  format  for  the  source  selection  plan,  its  size  and  detail  should 
reflect  file  complexity  of  the  acquisition.  You  should  include,  at  a  minimum,  a  discussion  of  the 
following: 

•  A  description  of  what  you  are  buying.  This  description  should  be  stated  in  functional 
terms  to  the  maximum  extent  possible  and  use  a  minimum  of  technical  language. 

•  A  description  of  the  evaluation  organization  structure.  It  may  be  helpful  to  include  -- 

An  organization  chart,  showing  the  evaluation  team’s  structure,  or  a  brief 
description  of  how  the  team  is  organized. 

The  duties  and  responsibilities  of  each  element  of  the  source  selection  team. 

The  evaluation  team’s  agenda  and  schedule. 

Information  on  the  need  for  preparation  and  training  of  the  evaluation  team. 

Security  procedures  to  be  used  by  the  evaluation  team  to  protect  classified, 
proprietary,  or  source  selection  information. 

CERTIFICATE  FOR  PERSONNEL  PARTICIPATING  IN  SOURCE  SELECTION 
CONCERNING  NONDISCLOSURE,  CONFLICTS  OF  INTEREST, 

AND  RULES  OF  CONDUCT 

Name: _  Organization: _ 

Title: _  Source  Selection: _ 

1.  I  acknowledge  that  I  have  been  selected  to  participate  in  the  source  selection  identified  above. 

I  certify  that  I  will  not  knowingly  disclose  any  contractor  bid  or  proposal  or  source  selection 
information  directly  or  indirectly  to  any  person  other  than  a  person  authorized  by  the  head  of  the 
agency  or  the  contracting  officer  to  receive  such  information.  I  understand  that  unauthorized 
disclosure  of  such  information  may  subject  me  to  substantial  administrative,  civil  and  criminal 
penalties,  including  fines,  imprisonment,  and  loss  of  employment  under  the  Procurement 
Integrity  Law  or  other  applicable  laws  and  regulations. 

2.  To  the  best  of  my  knowledge,  I  certify  that  neither  I  nor  my  spouse  nor  my  dependent 

children,  nor  members  of  my  household,  nor  personnel  with  whom  I  am  seeking  employment 
have  any  direct  or  indirect  financial  interest  in  any  of  the  firms  submitting  proposals,  or  their 
proposed  subcontractors  or  have  any  other  beneficial  interest  in  such  firm  except  as  fully _ 
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disclosed  on  an  attachment  to  this  certification. 

3.  I  certify  that  I  will  observe  the  following  rules  of  conduct: 

a.  I  will  not  solicit  or  accept,  directly  or  indirectly,  any  promise  of  future  employment  or 
business  opportunity  from,  or  engage,  directly  or  indirectly,  in  any  discussion  of  future 
employment  or  business  opportunity  with,  any  officer,  employee,  representative,  agent,  or 
consultant  of  a  competing  contractor. 

b.  I  will  not  ask  for,  demand,  exact,  solicit,  seek,  accept,  receive,  or  agree  to  receive,  directly  or 
indirectly,  any  money,  gratuity,  or  other  thing  of  value  from  any  officer,  employee, 
representative,  agent,  or  consultant  of  any  competing  offeror  for  this  acquisition.  I  will  advise 
my  family  that  the  acceptance  of  any  such  gratuity  may  be  imputed  to  me  as  a  violation,  and 
must  therefore  be  avoided. 

c.  I  will  not  discuss  evaluation  of  source  selection  matters  with  any  unauthorized  individuals 
(including  Government  personnel),  even  after  contract  award,  without  specific  prior  approval 
from  proper  authority. 

d.  I  understand  that  my  obligations  under  this  certification  are  of  a  continuing  nature.  If  at  any 
time  during  the  source  selection  process,  I  receive  a  contract  from  a  competing  contractor 
concerning  employment  or  other  business  opportunity,  the  offer  of  a  gift  from  a  competing 
contractor,  or  I  encounter  circumstances  where  my  participation  might  result  in  a  real,  apparent, 
or  potential  conflict  of  interest,  I  will  immediately  seek  the  advice  of  an  Ethics  Counselor  and 
report  the  circumstances  to  the  Source  Selection  Authority. 

I  understand  that  making  a  false,  fictitious,  or  fraudulent  certification  may  subject  me  to 
prosecution  under  Title  18,  United  States  Code,  Section  1001. 

Signature: _ _  Date: _ 


Blueprint:  Certifications 

•  Plans  for  presolicitation  activities  such  as  issuing  a  draft  solicitation  and  holding  a 
presolicitation  and/or  preproposal  conference  or  Advance  Planning  Briefing  for 
Industry. 

•  An  acquisition  strategy  summary  that  includes  an  explanation  of  the  contract  type  to 
be  used  (e.g.,  firm  fixed  price). 

•  The  proposed  evaluation  factors  and  subfactors,  their  relative  importance,  and 
associated  evaluation  standards. 

•  A  description  of  the  evaluation  process  you  are  using  (i.e.,  lowest  price  technically 
acceptable,  tradeoff,  or  hybrid)  and  any  innovative  techniques  such  as  multiple 
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phases  or  oral  presentations,  or  tailoring.  See  Appendix  E  for  details  on  oral 
presentations.  Include  a  description  of  the  rating  system  you  are  using. 

•  A  schedule  of  significant  milestones  that  should  cover,  at  a  minimum,  the  period 

beginning  with  the  designation  of  the  source  selection  authority  and  continuing 
through  the  period  from  receipt  of  proposals  through  the  signing  of  the  contract, 
during  which  evaluation,  negotiation,  and  selection  take  place. 

Selecting  Evaluation  Factors  and  Subfactors 

You  must  clearly  state  in  the  solicitation  and  source  selection  plan  all  the  evaluation  factors  and 
subfactors  that  you  will  consider  in  making  the  source  selection  and  their  relative  importance. 
These  factors  and  subfactors  inform  offerors  of  all  the  significant  considerations  in  selecting  the 
best  value  source  and  the  relative  importance  the  Government  attaches  to  each  of  these 
considerations.  Offerors  should  understand  the  basis  upon  which  their  proposals  will  be 
evaluated  and  how  they  can  best  prepare  their  proposals. 


Evaluation  factors  help  offerors 
understand  the  evaluation  process 


Structure  evaluation  factors  and  subfactors  and  their  relative  order 
of  importance  to  clearly  reflect  the  Government’s  need  and facilitate 
preparation  of  proposals  that  best  satisfy  that  need. 

A  multidisciplined  team  chooses  the  evaluation  factors  and  subfactors  based  on  user 
requirements,  acquisition  objectives,  perceived  risks,  and  thorough  market  research.  Thorough 
research  of  the  market  helps  the  team  identify  the  capabilities  of  different  industry  sectors  and 
where  those  capabilities  are  most  likely  to  differ  among  potential  offerors.  The  team  then  selects 
only  those  factors  that  will  help  differentiate  among  offerors  and  surface  the  most  advantageous 
proposal. 


Limit  evaluation  factors  and  subfactors  to  those  areas  that  will  reveal 
substantive  differences  or  risk  levels  among  competing  proposals. 


Limit  evaluation  factors 
to  true  discriminators 


Cost  Factors 

The  Competition  in  Contracting  Act  (CICA),  as  implemented  in  the  FAR,  requires  that  price  or 
cost  to  the  Government  be  included  as  an  evaluation  factor  in  every  source  selection.  This  is 
because  affordability  must  always  be  a  consideration  when  spending  taxpayer  dollars. 
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Always  include  cost  or  price 
as  an  evaluation  factor 


The  relative  importance  between  cost  or  price  and  the  noncost  factors  must  also  be  reflected  in 
both  the  solicitation  and  the  weights  or  priority  statements  in  the  source  selection  plan. 

However,  cost/price  is  not  numerically  scored  in  the  evaluation  of  proposals,  because  of  possible 
distortions  that  can  result  when  arbitrary  methods  are  used  to  convert  cost/price  into  scores. 

Cost-related  factors  and  considerations  will  vary  depending  on  the  type  of  contract.  Regardless 
of  contract  type,  reasonableness  must  always  be  a  consideration,  as  the  FAR  requires  that 
contracts  be  awarded  only  at  prices  or  costs  that  are  fair  and  reasonable. 

Cost  realism  plays  an  important  role  in  many  source  selections.  A  cost  realism  analysis  is  an 
independent  review  of  each  offeror’s  cost  proposal  to  determine  if  specific  estimated 

proposed  cost  elements  are  realistic  for  the  work  to  be  performed;  reflect  a  clear  understanding 
of  the  requirements;  and  are  consistent  with  the  unique  methods  of  performance  and  materials  in 
the  offeror’s  technical  proposal. 

Cost  realism  must  be  considered  when  a  cost  reimbursement  contract  is  anticipated.  Under  a 
cost  type  contract,  the  proposed  cost  estimates  may  not  be  valid  indicators  of  final  actual  costs 
that  the  Government  will  be  obligated  to  pay.  For  this  type  of  contract,  a  cost  realism  analysis  is 
performed  and  used  to  determine  the  probable  cost  of  performance  for  each  offeror.  Selection 
decisions  should  be  based  on  these  probable  cost  estimates.  Significant  differences  between 
proposed  and  most  probable  costs  may  signal  increased  performance  risks. 

Cost  realism  may  also  be  considered  for  fixed  price  incentive  contracts  or,  in  exceptional  cases, 
for  other  fixed  price  type  contracts  especially  when  there  are  concerns  that  offerors  may  try  to 
“buy  in”  or  where  other  complexities  of  the  acquisition  could  result  in  misunderstanding  the 
requirements.  In  such  cases,  a  cost  realism  analysis  may  be  useful  for  determining  if  there  is  a 
significant  risk  of  future  performance  because  of  unrealistically  high  or  low  prices.  However, 
proposed  fixed  prices  are  not  adjusted  for  cost  realism  during  the  evaluation. 

The  solicitation  must  clearly  state  what  costs  will  be  evaluated.  These  costs  may  include  costs 
for  the  basic  effort  only,  basic  plus  all  options,  or  costs  incurred  as  a  result  of  acquiring  or 
owning  an  item  (e.g.,  transportation,  life  cycle  costs).  The  solicitation  should  also  clearly 
indicate  to  offerors  how  the  cost  factor  will  be  assessed  for  that  acquisition. 

Past  Performance 

The  caliber  of  a  contractor’s  performance  on  previous  contracts  shall  be  included  as  an 
evaluation  factor  in  competitively  negotiated  acquisitions  unless  the  contracting  officer 
documents  why  it  would  not  be  appropriate  for  the  specific  circumstances  of  the  acquisition.  A 
thorough  evaluation  of  past  performance,  to  include  information  that  is  outside  of  the  offerors’ 
proposals,  serves  to  ensure  that  awards  are  made  to  good  performers  rather  than  to  just  good 
proposal  writers.  See  Appendix  D  for  details  on  evaluating  past  performance. 
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Technical  Factors 


Technical  evaluation  factors  address  the  proposal’s  technical  and  performance  efficiency.  These 
factors  may  include  such  considerations  as  technical  approach  and  capabilities,  management 
approach  and  capabilities,  experience  and  personnel  qualifications  relative  to  satisfying  critical 
aspects  of  the  government’s  requirements.  Technical  factors  must  be  developed  specifically  for 
each  acquisition,  taking  into  consideration  the  particular  objectives  and  requirements  of  the 
acquisition.  These  factors  should  be  those  discriminators  that  are  determined  after  thorough 
market  research  as  most  likely  to  reveal  substantive  differences  in  technical  approaches  or  risk 
levels  among  competing  proposals. 

The  source  selection  team  has  broad  discretion  in  determining  the  technical  evaluation  factors 
and  subfactors,  their  relative  importance,  and  the  way  in  which  they  will  be  applied. 

However,  too  many  factors  and  subfactors  can  lead  to  a  leveling  of  ratings,  in  which  the  final 
result  may  be  a  number  of  closely  rated  proposals  with  little  discrimination  among  competitors. 

It  is  not  the  number  of  noncost  factors  that  is  critical,  but  having  the  right  factors. 

Basic  requirements  for  noncost  evaluation  factors  are: 

•  A  reasonable  expectation  of  variance  among  proposals  in  that  area. 

•  A  variance  that  you  can  measure  either  quantitatively  or  qualitatively. 

•  The  factor  must  be  a  true  discriminator. 

An  evaluation  factor  should  be  chosen  only  if  your  requirements  warrant  a  comparative 
evaluation  of  that  area.  The  simplest  way  to  assess  a  potential  evaluation  factor  is  to  ask:  “Will 
superiority  in  this  factor  provide  value  to  the  Government  and  is  the  Government  willing  to  pay 
more  for  that  superiority?” 


Best  Practices 

Selecting  the  right  evaluation  factors  is  one  of  the  most  important  decisions  you  will  make  in 
designing  your  evaluation  process.  We  are  often  faced  with  the  triple  problems  of  less  time,  less 
funds,  and  fewer  available  personnel  to  devote  to  source  selections.  If  you  don’t  concentrate  on 
what’s  important  in  selecting  the  best  value  offeror  you  could  end  up  with  the  evaluation  team 
wasting  a  lot  of  time  and  effort  looking  at  issues  that  don’t  differentiate  between  offerors.  This 
can  also  result  in  a  weak  evaluation  that  doesn’t  give  the  source  selection  authority  the 
information  needed  to  make  a  good  selection. 

There  are  certain  factors  that  you  must  consider  in  any  competitive  source  selection.  Price/cost 
is  an  automatic  factor  that  you  always  have  to  consider.  You  also  have  to  consider  past 
performance  in  your  evaluation  process  unless  the  contracting  officer  documents  why  it  is  not 
appropriate  for  the  specific  circumstances  of  the  acquisition.  In  addition,  you  may  have  to  add 
factors  that  are  required  by  regulation  for  specific  acquisitions,  such  as  any  applicable 
preferences  for  small  entities.  From  here,  you  add  other  factors  and  subfactors  that  are  important 
to  deciding  which  is  the  most  advantageous  proposal.  Remember,  not  everything  that  the  offeror 
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has  to  do  under  the  contract  is  really  a  discriminator  that  will  help  you  decide  which  proposal 
will  result  in  the  best  value.  Consider  what  you  are  buying  and  what  will  really  discriminate. 

How  to  select  the  additional  factors/subfactors?  Consider  the  following  methodology: 

•  Research  the  market  for  what  you  are  buying  and  your  probable  universe  of  offerors. 

•  Form  an  Integrated  Product  Team  (BPT)  and  brainstorm  critical  factors  and 
subfactors. 

•  Select  only  those  factors  and  subfactors  likely  to  surface  the  most  advantageous 
proposals. 

•  Define  the  key  discriminators  and  prioritize  the  list. 

•  Get  source  selection  authority  approval  of  the  list  of  factors/subfactors. 

•  Clearly  and  concisely  tell  offerors  in  the  solicitation  what  the  factors/subfactors  are 
and  their  relative  importance. 

•  Listen  carefully  to  industry  feedback  from  presolicitation  exchanges  to  see  if  your 
choices  are  right.  If  necessary,  change  the  factors/subfactors  before  solicitation. 

Weighting  the  Factors  and  Subfactors 

After  determining  the  evaluation  factors  and  subfactors,  their  relative  importance  to  each 

other  must  be  established.  The  relative  importance  of  factors  and  subfactors  must  be  consistent 
with  the  stated  solicitation  requirements.  If  their  relative  importance  does  not  accurately  reflect 
the  Government’s  requirements  and  objectives,  the  source  selection  authority  may  later  award  to 
an  offeror  whose  proposal  may  not  be  the  best  value.  As  a  general  rule,  the  higher  the  technical 
or  performance  risk,  the  greater  the  emphasis  on  noncost  factors.  The  relative  importance 
between  all  noncost  factors  combined  and  cost  or  price  must  also  be  described  using  the  terms, 
“significantly  more  important,”  “approximately  equal,”  or  “significantly  less  important.”  This 
relative  ranking  must  be  reflected  in  both  the  solicitation  and  the  weights  or  priority  statements 
in  the  source  selection  plan. 

The  relative  importance  of  evaluation  factors  and  subfactors  is  usually  established  by  priority 
statements,  numerical  weighting,  or  a  combination  of  these. 

•  Priority  or  tradeoff  statements,  numerical  weighting,  or  a  combination  of  these 
usually  establishes  the  relative  importance  of  evaluation  factors  and  subfactors. 

•  Priority  or  tradeoff  statements  would  relate  one  factor  to  others.  For  example,  in  a 
priority  statement,  the  cost/price  factor  may  be  said  to  be  slightly  more  important 
than  a  noncost  factor  called  “performance  risk”  but  slightly  less  important  than  a 
noncost  factor  called  “technical  merit.” 
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•  Numerical  weighting  would  involve  assigning  relative  importance  to  the  factors  and 
subfactors  using  points  or  percentages.  Although  numerical  weights  may  be  used  in 
making  the  tradeoff  analysis  and  decision,  the  weights  themselves  may,  but  need  not 
be  disclosed  in  the  solicitation.  If  you  don’t  disclose  the  numerical  weights 
themselves  in  the  solicitation,  they  must  be  described  in  terms  of  priority  or  tradeoff 
statements. 

Cost/price  as  an  evaluation  factor  is  never  scored  or  rated  as  part  of  the  evaluation.  But,  just  like 
all  the  other  factors  and  subfactors,  cost/price  has  to  be  weighted  to  indicate  its  importance 
relative  to  the  other  evaluation  factors  and  subfactors  and  the  overall  evaluation.  The  weight 
given  to  cost/price  reflects  its  relative  importance  in  selecting  the  best  proposal  for  award.  The 
circumstances  of  your  particular  acquisition  will  indicate  how  important  cost/price  is  in 
satisfying  your  requirement. 


Developing  Evaluation  Standards 

Evaluators  must  be  able  to  determine  the  relative  merit  of  each  proposal  with  respect  to  the 
evaluation  factors.  Evaluation  standards  provide  guides  to  help  evaluators  measure  how  well  a 
proposal  addresses  each  factor  and  subfactor  identified  in  the  solicitation.  Standards  permit  the 
evaluation  of  proposals  against  a  uniform  objective  baseline  rather  than  against  each  other.  The 
use  of  evaluation  standards  minimizes  bias  that  can  result  from  an  initial  direct  comparison  of 
proposals.  Standards  also  promote  consistency  in  the  evaluation  by  ensuring  that  the  evaluators 
evaluate  each  proposal  against  the  same  baseline.  In  developing  standards  for  each  evaluation 
factor  and  subfactor,  you  should  consider  the  following: 

•  As  you  develop  your  evaluation  factors,  concurrently  draft  a  standard  for  each  factor 
and  subfactor. 

•  Define  the  standard  by  a  narrative  description  that  specifies  a  target  performance 
level  that  the  proposal  must  achieve  in  order  to  meet  the  standard  for  the  factor  or 
subfactor  consistent  with  the  requirements  of  the  solicitation. 

•  Describe  guidelines  for  higher  or  lower  ratings  compared  to  the  standard  “target.” 

•  Overly  general  standards  should  be  avoided  because  they  make  consensus  among 
evaluators  more  difficult  to  obtain  and  may  obscure  the  differences  between 
proposals.  A  standard  should  be  worded  so  that  mere  inclusion  of  a  topic  in  an 
offeror’s  proposal  will  not  result  in  a  determination  that  the  proposal  meets  the 
standard.  (An  example  is  shown  at  Appendix  C.) 

•  While  it  is  sometimes  easier  to  develop  quantitative  standards  because  of  their 
definitive  nature,  qualitative  standards  are  commonly  used  in  source  selections. 
Standards,  as  part  of  the  source  selection  methodology,  should  be  included  in  the 
source  selection  plan. 

Establishing  a  Rating  Method 
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Typical  Ratings  and  Descriptors 

Each  rating  must  have  a  definition. 

TECHNICAL  MERIT  ratings  reflect  the  government’s  confidence  in  each  offeror’s  ability,  as 
demonstrated  in  its  proposal,  to  perform  the  requirements  stated  in  the  RFP.  Choose  one  method 
(e.g.,  numerical,  adjectival,  or  color)  to  evaluate  technical  merit. 

NUMERICAL 

ADJECTIVAL  COLOR 

DEFINITION 

90-100 

Excellent  Blue 

Proposal  demonstrates  excellent  understanding 
of  requirements  and  approach  that  significantly 
exceeds  performance  or  capability  standards. 

Has  exceptional  strengths  that  will 
significantly  benefit  the  Government. 

80-89 

Good  Green 

Proposal  demonstrates  good  understanding  of 
requirements  and  approach  that  exceeds 
performance  or  capability  standards.  Has  one 
or  more  strengths  that  will  benefit  the 
Government. 

70-79 

Satisfactory  Yellow 

Proposal  demonstrates  acceptable 
understanding  of  requirements  and  approach 
that  meets  performance  or  capability  standards. 
Acceptable  solution.  Few  or  no  strengths. 

60-69 

Marginal  Amber 

Proposal  demonstrates  shallow  understanding 
of  requirements  and  approach  that  only 
marginally  meets  performance  or  capability 
standards  necessary  for  minimal  but  acceptable 
contract  performance. 

<60 

Unsatisfactory  Red 

Fails  to  meet  performance  or  capability 
standards.  Requirements  can  only  be  met  with 
major  changes  to  the  proposal. 

PROPOSAL  RISK  ratings  assess  the  risks  and  weaknesses  associated  with  each  offeror’s 
proposed  approach  to  performing  the  requirements  stated  in  the  RFP.  It  is  an  overall  assessment 
derived  from  the  technical  evaluation  and  is  driven  by  each  of  the  subfactors  within  the  technical 
factor. 

ADJECTIVE 

DESCRIPTION 

Low  Risk 

Any  proposal  weaknesses  have  little  potential  to  cause  dismption  of  schedule, 
increase  in  cost,  or  degradation  of  performance.  Normal  contractor  effort  and 
normal  Government  monitoring  will  probably  minimize  any  difficulties. 
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Moderate  Risk 

Approach  has  weaknesses  that  can  potentially  cause  some  disruption  of 
schedule,  increase  in  cost,  or  degradation  of  performance.  However,  special 
contractor  emphasis  and  close  Government  monitoring  will  probably 
minimize  difficulties. 

High  Risk 

Approach  has  weaknesses  that  have  the  potential  to  cause  serious  disruption 
of  schedule,  increase  in  cost,  or  degradation  of  performance  even  with  special 
contractor  emphasis  and  close  Government  monitoring. 

PERFORMANCE  RISK  (Past  Performance)  ratings  assess  the  risks  associated  with  each 
offeror’s  likelihood  of  success  in  performing  the  requirements  stated  in  the  RFP  based  on  that 
offeror’s  demonstrated  performance  on  recent,  relevant  contracts. 

ADJECTIVE 

DESCRIPTION 

Very  Low  Risk 

Offeror’s  past  performance  record  provides  essentially  no  doubt  that  the 
offeror  will  successfully  perform  the  required  effort. 

Low  Risk 

Offeror’s  past  performance  record  provides  little  doubt  that  the  offeror  will 
successfully  perform  the  required  effort. 

Moderate  Risk 

Offeror’s  past  performance  record  provides  some  doubt  that  the  offeror  will 
successfully  perform  the  required  effort. 

High  Risk 

Offeror’s  past  performance  record  provides  substantial  doubt  that  the  offeror 
will  successfully  perform  the  required  effort. 

Very  High  Risk 

Offeror’s  past  performance  record  provides  extreme  doubt  that  the  offeror 
will  successfully  perform  the  required  effort. 

Unknown  Risk 

The  offeror  has  no  relevant  performance  record.  A  thorough  search  was 
unable  to  identify  any  past  performance  information. 

COST  -  NOT  “RATED.”  Reflects  the  evaluated  cost.  RFP  must  describe  method  by  which  cost 
will  be  evaluated  (e.g.,  how  probable  cost  or  life  cycle  cost  will  be  evaluated.) 

Blueprint:  Typical  Ratings  and  Descriptors 

A  rating  system  uses  a  scale  of  words,  colors,  numbers  or  other  indicators  to  denote  the  degree  to 
which  proposals  meet  the  standards  for  the  noncost  evaluation  factors.  Thus,  a  rating  system 
helps  evaluators  assess  a  proposal’s  merit  with  respect  to  the  evaluation  factors  and  subfactors  in 
the  solicitation.  Some  commonly  used  rating  systems  are  adjectival,  color  coding,  and 
numerical. 

Rating  systems  which  use  adjectives  or  colors  are  usually  the  most  successful  because  they  allow 
maximum  flexibility  in  making  the  tradeoffs  among  the  evaluation  factors.  A  narrative 
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definition  must  accompany  each  rating  in  the  system  so  that  evaluators  have  a  common 
understanding  of  how  to  apply  the  rating.  For  example,  a  rating  of  excellent  (or  blue  or  90-100) 
could  be  defined  as  meaning  an  outstanding  approach  to  specified  performance  with  a  high 
probability  of  satisfying  the  requirement.  What  is  key  in  using  a  rating  system  in  proposal 
evaluations,  is  not  the  method  or  combination  of  methods  used,  but  rather  the  consistency  with 
which  the  selected  method  is  applied  to  all  competing  proposals  and  the  adequacy  of  the 
narrative  used  to  support  the  rating. 


Adjectival 

Adjectives  (such  as  excellent,  good,  satisfactory,  marginal,  and  unsatisfactory)  are  used  to 
indicate  the  degree  to  which  the  offeror’s  proposal  has  met  the  standard  for  each  factor 
evaluated.  Adjectival  systems  may  be  employed  independently  or  in  connection  with  other 
rating  systems. 


Color  Coding 

This  system  uses  colors  to  indicate  the  degree  to  which  the  offeror’s  proposal  has  met  the 
standard  for  each  factor  evaluated.  For  instance,  the  colors  blue,  green,  yellow,  amber,  and  red 
may  indicate  excellent,  good,  satisfactory,  marginal,  or  unsatisfactory  degrees  of  merit, 
respectively. 


Numerical 

This  system  assigns  point  scores  (such  as  0-10  or  0-100)  to  rate  proposals.  This  rating  system 
generally  allows  for  more  rating  levels  and  thus  may  appear  to  give  more  precise  distinctions  of 
merit.  However,  numerical  systems  can  have  drawbacks  as  their  apparent  precision  may  obscure 
the  strengths,  weaknesses,  and  risks  that  support  the  numbers.  Therefore,  some  organizations  do 
not  permit  the  use  of  numerical  rating  systems. 

Narrative 

Narrative  is  used  in  conjunction  with  a  rating  system  to  indicate  a  proposal’s  strengths, 
weaknesses,  and  risks.  Adjectival,  color,  and  numerical  ratings  must  be  supported  with  narrative 
statements.  Narrative  statements  can  describe  the  proposals’  relative  strengths,  weaknesses,  and 
risks  to  the  source  selection  authority  in  a  way  that  adjectives,  colors,  and  numbers  alone  cannot. 
A  narrative  is  required  when  evaluation  standards  are  being  applied,  when  a  comparison  of 
proposals  is  being  made,  and  when  a  cost/technical  tradeoff  is  conducted.  The  narrative  provides 
a  reasonable  and  rational  basis  for  the  selection  decision. 
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The  Solicitation 


Ensure  consistency  among  the  objectives  of  the  acquisition, 
the  contracting  strategy,  the  evaluation  plan,  the  solicitation, 
the  evaluation  and  selection. 

All  the  parts  of  the  solicitation  work  together  to  communicate  government  requirements  to 
potential  offerors.  The  solicitation  provides  all  the  information  the  offeror  needs  to  understand 
what  you  are  buying,  how  you  are  buying  it,  and  how  you  will  select  who  to  buy  it  from.  This 
information  includes:  the  work  requirements;  the  terms  and  conditions;  evaluation  factors  and 
significant  subfactors;  the  relative  importance  of  the  factors  and  subfactors;  instructions  to 
offerors,  including  whether  award  might  be  made  without  discussions;  and  other  exhibits  and 
attachments.  When  read  as  a  whole,  the  solicitation  should  convey  to  the  offerors  a  clear 
understanding  of  what  you  are  buying  and  the  areas  where  technical  and  cost  tradeoffs  can  be 
made  in  their  proposals  to  best  satisfy  the  Government  requirements. 

Industry  frequently  complains  that  solicitations  have  major  conflicts.  Particularly  troublesome 
are  conflicts  among  the  descriptions  of  what  we’re  buying,  instructions  on  how  to  prepare  a 
proposal,  and  guidance  on  important  factors/subfactors  and  the  ground  rules  for  the  evaluation. 
An  inconsistent  solicitation  may  result  when  different  groups  of  people  develop  the  different 
sections  without  proper  coordination.  Such  a  solicitation  can  defeat  our  objectives,  cause 
unnecessary  delays,  or  lead  to  litigation. 

Coordination  within  a  multidisciplined  acquisition  team,  whose  members  are  stakeholders  in  the 
acquisition  and  have  a  commitment  to  work  together,  is  the  best  way  to  ensure  consistency.  You 
may  also  find  it  beneficial  to  develop  a  matrix  that  correlates  the  solicitation  sections  and  content 
to  ensure  solicitation  consistency.  You  may  want  to  provide  industry  with  a  copy  of  the  matrix 
as  a  reference  tool  to  aid  in  proposal  preparation.  This  approach  promotes  understanding  of  the 
linkage  within  the  solicitation  and  explains  how  all  parts  of  the  proposal  will  be  used  in  the 
evaluation  process. 

Appendix  C  illustrates  how  the  key  solicitation  documents  and  evaluation  standards  track  to  one 
another  and  shows  the  recommended  sequencing  for  document  preparation. 

Another  way  to  promote  understanding  of  the  solicitation  is  to  foster  a  presolicitation  dialogue 
with  industry. 

This  can  be  accomplished  through  use  of  various  communication  forums  such  as  Commerce 
Business  Daily  notices.  Advance  Planning  Briefings  for  Industry,  draft  solicitations,  and/or 
presolicitation/preproposal  conferences. 

Performance  Requirements  --  (Specifications,  Work  Statement,  or  Equivalent) 

The  way  you  present  the  Government’s  requirements  in  the  solicitation  can  have  a  significant 
impact  upon  a  source  selection  using  the  tradeoff  approach.  For  example,  use  of  a  work 
breakdown  structure  (WBS)  in  the  work  statement  for  the  most  complex  cost  type  contracts  can 
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help  ensure  offerors’  pricing  breakdowns  are  consistent  and  comparable.  Some  additional  areas 
to  consider  when  preparing  the  work  requirements  for  the  solicitation  include: 


Functional  or  Performance  Requirements 

Use  functional  or  performance  requirements  to  the  maximum  extent  possible.  In  some  cases,  it 
may  be  more  difficult  to  develop  evaluation  standards  and  conduct  the  evaluation  process  itself; 
however,  there  are  benefits  to  using  functional  or  performance  requirements.  These  benefits 
include: 

•  Increased  competition. 

•  Access  to  the  best  commercial  technology. 

•  Better  technical  solutions  for  better  prices  as  a  result  of  offeror  innovation. 

•  Functional  or  performance  requirements  can  usually  be  developed  faster  than  design 
requirements. 

•  Fewer  situations  may  exist  for  protests. 


Using  performance  requirements 
can  lead  to  offeror  innovation 


Design  Requirements 

You  should  limit  the  number  of  design  requirements  to  those  essential  to  meet  mission  needs. 
Design  requirements  may: 

•  Limit  competition. 

•  Limit  situations  where  potential  offerors  can  propose  innovative  solutions. 

•  Slow  the  specification  development  process. 

•  Provide  more  situations  for  an  offeror  to  protest  (e.g.,  because  of  the  belief  that  the 
winning  proposal  did  not  meet  all  the  minimum  requirements  or  that  the 
requirements  were  unnecessarily  restrictive  of  competition). 

Proposal  Submission  Information 

The  instructions  for  preparing  and  submitting  proposals  are  critical  to  an  acquisition  using  the 
tradeoff  approach.  There  has  to  be  a  linkage  between  solicitation  requirements,  each  evaluation 
factor  and  subfactor  and  the  proposal  preparation  instructions. 
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Each  evaluation  factor  and  subfactor 
must  correlate  directly  with  the 
proposal  preparation  instructions 


If  you  cannot  cross-walk  the  solicitation  requirements,  factors/subfactors  and  the  proposal 
instructions,  you  have  a  conflict  that  you  need  to  correct. 

Request  only  the  information  needed  to  evaluate  proposals 
against  the  evaluation  factors  and  subfactors .  Never  ask 
for  information  you  do  not  intend  to  evaluate. 

The  information  requested  from  offerors  must  correlate  with  the  evaluation  factors  and 
subfactors.  However,  instructions  that  require  voluminous  information  can  cause  potential 
offerors  to  forego  responding  to  the  solicitation  in  favor  of  a  less  costly  business  opportunity. 
Furthermore,  excessive  size  of  proposals  may  increase  the  Government’s  costs  to  perform  the 
evaluation  and  length  of  the  evaluation  period.  In  order  to  simplify  the  preparation  of  proposals 
and  to  make  the  evaluation  easier,  you  may  wish  to  consider  imposing  a  realistic  limit  on  the 
number  of  pages  and  foldouts  to  be  submitted. 

The  instructions  on  the  preparation  and  submission  of  proposals  must: 

•  Be  clearly  and  precisely  stated. 

•  Be  keyed  to  the  evaluation  factors  and  subfactors. 

•  Describe  the  type,  scope,  content,  and  format  of  the  information  to  be  submitted. 

•  Describe  the  order  in  which  proposal  responses  and  materials  are  to  appear. 

•  Be  limited  to  the  information  needed  to  do  the  evaluation. 

•  Properly  written  proposal  preparation  instructions  simplify  the  evaluators’  job.  That 
is,  evaluators  do  not  have  to  learn  a  new  format  for  each  proposal;  they  can  evaluate 
the  same  requirements  in  each  proposal  in  the  same  way.  With  a  sufficient  degree  of 
structure  in  the  proposal  preparation  requirements,  you  may  be  able  to  accept 
proposals  in  electronic  form  and  use  some  automation  in  the  evaluation  process. 

Proposal  Evaluation  Information 

Clearly  state  in  the  solicitation  the  basis  upon  which  the 
Government  will  make  the  source  selection  decision. 

The  information  from  the  Source  Selection/Evaluation  Plan  that  you  provide  in  the  solicitation 
on  evaluation  factors  and  subfactors  and  their  relative  importance  forms  the  basis  for  evaluating 
offerors’  proposals  and  making  the  cost/technical  tradeoff.  The  solicitation  is  the  official  vehicle 
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for  you  to  communicate  to  offerors  which  factors  and  subfactors  or  ground  rules  the  Government 
will  use  to  select  the  most  advantageous  proposal  for  award. 

Consider  the  following  points  in  designing  the  solicitation: 

•  Provide  the  evaluation  factors  and  subfactors  verbatim  from  the  source  selection 
plan. 

•  Provide  the  actual  numerical  weights  at  the  factor  level. 

•  Provide  an  estimate  of  what  you’ve  identified  as  an  affordable  target  price  range  for 
the  acquisition,  based  on  your  market  research  or  other  reviews. 

This  information  can  help  offerors  to  better  focus  on  those  aspects  of  the  mission  objectives 
where  additional  value  can  be  important  and  to  better  respond  to  the  Government’s  needs  by 
giving  emphasis  to  those  things  most  important  to  the  Government.  To  reap  the  benefits  of 
better  proposals  you  need  to  include  and  adequately  describe  all  the  factors  and  subfactors  (as 
reflected  in  the  source  selection  plan)  that  will  be  considered  in  making  the  selection. 

The  solicitation  must  also  inform  offerors  of  any  minimum  requirements  that  apply  to  particular 
evaluation  factors  and  subfactors  that  have  to  be  met.  You  need  to  distinguish  between 
minimum  acceptable  requirements  and  desirable  objectives  or  features  that  you  would  be  willing 
to  pay  extra  for.  If  you  elect  to  include  desirable  objectives  or  features  in  addition  to  minimum 
requirements,  the  solicitation  must  clearly  explain  how  you  will  evaluate  them  and  whether  or 
not  credit  will  be  given  in  the  evaluation  for  exceeding  such  desirables. 
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Evaluation  Considerations 


Overview 


The  source  selection  evaluation  process  includes  examining  each  proposal  in  detail  against  the 
evaluation  factors  and  subfactors  and  the  requirements  set  forth  in  the  solicitation,  and  assigning 
a  rating,  with  a  supporting  narrative.  The  proposal  evaluation  process  assesses  the  proposal  and 
the  offeror’s  ability  to  perform.  At  this  stage,  it  does  not  analyze  proposals  against  each  other 
and  it  must  be  conducted  in  a  fair,  comprehensive,  and  impartial  manner. 


Evaluations  must  be  fair, 
thorough,  and  impartial 


The  evaluation  process  can  be  complicated  no  matter  how  much  planning  and  tailoring  you  do. 
You  might  find  through  your  market  research  that  you  are  going  to  receive  many  proposals  — 
good  for  competition  but  a  situation  that  could  drag  out  evaluation.  You  might  explore  the  world 
of  automated  source  selection  tools  that  can  increase  your  efficiency.  There  are  commercial 
packages  available  and  some  activities  have  developed  packages  in  house  that  you  could  use. 
Evaluators  still  have  to  evaluate  each  proposal,  but  these  tools  might  ease  the  administrative 
burden  that  comes  with  a  great  number  of  proposals. 

Reasons  for  the  Evaluation  Process 


The  principal  purposes  of  the  process  are  to: 

•  Determine  which  proposals  are  acceptable  and/or  within  the  competitive  range. 

•  Provide  a  sound  basis  for  the  source  selection  authority  to  make  an  informed  and 
reasoned  selection  by: 

Presenting  a  clear  picture  of  the  issues  considered  during  evaluation  by 
identifying  areas  of  uncertainty  as  well  as  those  which  provide  substantial 
assurance  of  a  successful  outcome. 

Listing  the  strengths,  weaknesses,  and  risks  of  the  proposed  approaches. 

Evaluation  Process  Tasks 

Evaluation  tasks  will  vary  in  number  and  content  with  each  source  selection.  However,  several 
especially  important  tasks  are  discussed  below. 

Familiarization 


141 


Prior  to  receipt  of  proposals,  each  evaluator  should  become  familiar  with  the  solicitation’s 
requirements,  the  source  selection  plan,  and  the  rating  system.  You  should,  especially  for  those 
evaluators  with  no  prior  source  selection  evaluation  experience,  conduct  training  that  includes  an 
overview  of  the  solicitation  and  of  the  work  expected  throughout  the  source  selection  process. 
The  training  should  include  how  to  properly  document  each  proposal’s  strengths,  weaknesses, 
and  risks. 


Cost  Evaluations 

Cost  or  price  must  be  an  evaluation  factor  in  all  acquisitions.  The  cost  evaluation  will  vary 
depending  on  the  specific  circumstances  of  each  acquisition. 

For  fixed  price  contracts,  the  evaluation  normally  should  be  as  simple  as  a  comparison  of  the 
offered  prices  to  ensure  the  contract  price  is  fair  and  reasonable.  Other  techniques  of  price 
analysis  may  also  be  used.  Do  not  perform  a  cost  analysis  unless  there  is  no  other  way  to 
determine  if  the  price  of  the  otherwise  successful  offeror  is  reasonable. 

For  cost-reimbursement  contracts,  you  must  analyze  costs  for  both  realism  and  reasonableness. 
The  cost  realism  analysis  enables  you  to  determine  the  probable  cost  of  performance  for  each 
offeror.  This  precludes  an  award  decision  based  on  overly  optimistic  offeror’s  cost  estimates 
where  risks  of  an  overrun  may  be  significant. 

A  cost  realism  analysis  requires  an  independent  review  of  specific  elements  of  each  offeror’s 
proposed  cost  estimate  to  determine  whether  the  estimated  proposed  cost  elements  for  contract 
resources  (e.g.,  labor  and  material)  are  realistic,  show  understanding  of  the  work,  and  are 
consistent  with  the  demands  of  the  work  which  will  actually  be  required,  given  each  offeror’s 
unique  methods  of  performance  and  materials  described  in  their  technical  proposal. 

The  probable  cost  should  reflect  the  Government’s  best  estimate  of  the  cost  of  any  contract, 
which  is  most  likely  to  result  from  the  offeror’s  proposal.  This  estimate  is  determined  by 
adjusting  each  offeror’s  proposed  cost,  and  fee  when  appropriate,  upwards  or  downwards  to 
reflect  any  additions  or  reductions  in  personnel,  equipment,  or  materials  resulting  from  the  cost 
realism  assessment. 

For  the  cost  realism  evaluation  of  an  offeror’s  proposal,  you  have  to  decide  what  information  you 
need.  The  amount  and  type  of  information  will  vary  depending  on  the  circumstances  of  your 
acquisition.  You  may  have  to  get  more  after  you  start  evaluating  the  proposals.  However,  like 
other  proposal  requirements,  you  should  only  request  the  minimum  amount  of  information  that  is 
necessary.  Also,  remember  that  any  information  you  use  only  for  the  cost  realism  analysis  is  not 
considered  cost  or  pricing  data. 

To  the  extent  that  differences  between  proposed  costs  and  probable  costs  reflect  significant  risks 
of  future  performance  or  lack  of  understanding,  that  risk  or  lack  of  understanding  should  be 
reflected  in  the  noncost  evaluation.  In  such  cases,  you  should  also  seriously  consider  whether  or 
not  the  proposed  cost  and  fee  or  price  can  be  determined  fair  and  reasonable  to  both  parties. 

The  probable  cost  estimates  developed  for  each  offeror  are  used  to  evaluate  and  compare 
proposals  and  ultimately  to  select  the  proposal  expected  to  result  in  the  best  value. 
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Past  Performance  Evaluations 

Unless  you  are  using  a  lowest  price  technically  acceptable  approach,  the  past  performance 
evaluation  involves  a  comparative  assessment  of  performance  risk  associated  with  each  proposal. 
It  describes  the  degree  of  confidence  the  government  has  in  the  offeror’s  ability  to  perform  based 
on  that  offeror’s  demonstrated  record  of  past  and  present  work  similar  to  the  work  to  be 
performed.  If  properly  conducted,  the  past  performance  evaluation  and  the  preaward  survey  will 
complement  each  other  and  provide  a  more  complete  picture  of  an  offeror  than  either  one  could 
by  itself. 

Appendix  D  contains  procedures  for  evaluating  past  performance  in  source  selections,  including 
those  acquisitions  where  selection  is  based  solely  on  cost/price  and  past  performance. 

Technical  Evaluations 

PROPOSAL  EVALUATION  WORKSHEET 


RFPNo: 

EVALUATOR’S  NAME:  I  OFFEROR: 


FACTOR:  |  VOLUME/PARAGRAPH: 


SUBFACTOR:  I  PAGE  NUMBER: 


Merit  Ratings:  (Refer  to  your  rating  definitions, 


Evaluator’s  Rating:  (Merit/Risk) 


Initial  Rating:  (e.g.,  G/M)  Evaluator  Initials/  Date:  Team  Leader  Initials/Date: 

Discussions:  Evaluator  Initials/  Date:  Team  Leader  Initials/Date: 

Final  Rating:  Evaluator  Initials/  Date:  Team  Leader  Initials/Date: 

RATIONALE:  Include  supporting  rationale  for  the  ratings.  Using  the  evaluation  standards  and 
rating  definitions,  state  the  evaluation  results  in  terms  of  strengths;  weaknesses/significant _ 
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weaknesses/deficiencies;  and  risks.  Include  questions  that  address  ambiguities  or  other  concerns 
(e.g.,  perceived  deficiencies,  weaknesses,  omissions,  mistakes).  Identify  all  comments  and 
questions  below  with  the  rating:  (e.g.,  Initial  Rating  (IR),  Result  of  Discussions  (RD)  or  Final 
Rating  (FR).  Use  continuation  sheets  or  a  database  as  needed  and  a  separate  sheet  for  every 
factor  and  subfactor. 


STRENGTHS: 


WEAKNESSES/SIGNIFICANT  WEAKNESSES/DEFICIENCIES: 

(Precede  comment  with  an  [S]  if  it  identifies  a  significant  weakness  or  a  [D]  if  a  deficiency.) 


RISKS: 


QUESTIONS: 


Blueprint:  Typical  Evaluation  Worksheet 

Evaluators  must  examine  each  proposal  individually  in  detail  to  measure  it  against  the  evaluation 
factors  and  subfactors  in  the  solicitation.  Evaluators  ask  questions  such  as,  “How  much?”  or 
“How  well?”  assign  a  rating  and  document  the  basis  for  the  rating.  This  is  the  core  of  the 
evaluation  process. 

Normally,  technical  evaluations  should  be  conducted  independent  of  the  cost/price  evaluations  so 
that  technical  findings  and  conclusions  will  not  be  influenced  by  knowledge  of  the  offered  costs. 
However,  in  some  instances,  it  may  be  appropriate  to  give  the  entire  evaluation  team  access  to 
price/cost  information  to  ensure  the  best  possible  overall  evaluation  and  enhance  the  evaluation 
of  cost  realism.  Such  a  review  can  help  verify  perceived  technical  strengths,  weaknesses  or  risks 
and/or  ensure  consistency  between  the  cost/price  and  technical  segments  of  the  proposals. 

All  evaluators  must  have  the  required  functional  expertise  and  training  to  evaluate  the  particular 
area  of  the  proposal  to  which  they  are  assigned.  They  should  also  be  thoroughly  familiar  with 
the  solicitation  and  the  source  selection  plan. 

Identifying  Proposal  Ambiguities  and  Inadequate  Substantiation 
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Evaluators  should  first  document  problems  in  evaluating  a  proposal  because  its  language  is 
ambiguous,  its  meaning  is  unclear,  or  it  has  failed  to  respond  to  the  solicitation  instructions. 
Evaluators  should  also  identify,  in  writing,  instances  in  which  an  offeror  has  not  provided 
enough  information  to  evaluate  the  feasibility  and  merit  of  its  proposed  approach.  The 
Contracting  Officer  can  then  seek  amplification  and  additional  information  to  address  such 
issues. 


Identifying  Strengths,  Deficiencies,  Significant  Weaknesses,  and  Risks 

Evaluators  must  identify  and  document  the  strengths,  deficiencies,  significant  weaknesses,  and 
the  accompanying  risks  of  the  competing  proposals.  Proposals  that  materially  fail  to  meet  a 
Government  requirement  or  that  contain  a  combination  of  significant  weaknesses  that  increase 
the  risk  of  unsuccessful  performance  are  considered  to  be  deficient. 

Narrative  statements  must  be  used  to  establish  a  written  record.  Numerical  scores  and  other 
rating  techniques  are  not  conclusive  data  to  make  the  source  selection  decision.  Only 
evaluations  and  ratings  substantiated  by  specific  strengths,  weaknesses,  and  risks  can  be  credible 
and  justifiable.  General  terms  such  as  “weak,”  “poor,”  or  “excellent”  must  be  supported  with 
specific  reasons  as  to  why  the  proposal  is  “weak,”  “poor,”  or  “excellent”  in  relation  to  the 
standard  for  the  specific  factor  and  subfactor  being  evaluated. 

The  strengths,  weaknesses,  and  risks  of  each  proposal  form  a  large  part  of  the  basis  for  the 
source  selection  decision. 


Documenting  proposal  strengths, 
weaknesses,  and  risks  is  critical 


Generally,  the  fact  that  a  proposal  is  deficient  as  submitted  does  not  necessarily  mean  that  it  is 
excluded  from  further  consideration.  The  identification  of  these  vital  items  provides: 

•  An  element  for  the  contracting  officer  to  consider  in  determining  the  competitive 
range. 

•  The  framework  for  any  necessary  discussions  between  the  Government  and  the 
offeror. 

•  Specific  information  on  the  relative  strengths  and  weaknesses  of  competing 
proposals.  This  is  critical  to  the  successful  completion  of  an  acquisition  using  the 
tradeoff  approach  because  it  is  an  essential  element  of  the  evaluation  report  provided 
to  the  source  selection  authority. 

•  The  basis  for  tradeoff  analysis  ultimately  performed  by  the  source  selection  authority 
to  determine  if  differences  in  merit  between  proposals  justify  any  cost/price 
differential. 

•  The  framework  for  offeror  debriefings. 
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A  separate  evaluation  finding,  regardless  of  the  offered  cost  or  price,  can  be  a  determination  that 
a  proposal  is  technically  unacceptable.  This  finding  is  based  on  failure  to  meet  requirements,  or 
even  the  basic  intent  of  the  acquisition,  and  that  a  complete  revision  of  the  proposal  would  be 
required.  In  this  case,  you  would  be  put  in  the  position  of  leading  the  offeror  to  a  solution  or 
approach,  which  is  unfair  to  the  other  offerors. 

Consensus 

The  final  rating  of  each  proposal  should  be  assigned  by  consensus  of  the  evaluators.  Simple 
averaging  of  individual  evaluation  results  does  not  constitute  consensus.  Consensus  requires  a 
meeting  of  the  minds  on  classifications,  deficiencies,  strengths,  weaknesses,  and  risks.  In 
exceptional  cases  where  the  evaluators  are  unable  to  reach  agreement  without  unreasonably 
delaying  the  acquisition  process,  the  evaluation  report  may  include  the  majority  conclusion  and 
the  dissenting  view(s),  each  with  a  supporting  rationale. 

Exchanging  Information  with  Offerors 

Dialogue  with  offerors  after  receipt  of  proposals  allows  us  to  get  information  we  need  to  better 
understand  proposals  and  make  best  value  decisions.  While  all  such  dialogue  must  be  conducted 
in  a  fair  and  impartial  manner,  its  nature  and  extent  will  vary  depending  upon  when  it  occurs 
after  receipt  of  proposals. 

Who  is  in  Charge? 

The  contracting  officer  remains  the  focal  point  for  all  information  exchanges  with  prospective 
contractors  from  release  of  a  solicitation  through  contract  award.  Once  proposals  are  received, 
the  contracting  officer  also  controls  all  exchanges  with  offerors. 

Establishing  the  Ground  Rules 

Before  exchanging  any  information  with  offerors,  the  contracting  officer  should  ensure  that  team 
members  who  may  participate  in  such  exchanges  receive  instructions  not  to: 

•  Favor  one  offeror  over  another  (i.e.,  provide  the  offeror  with  suggested  ways  to 
correct  its  proposal  relative  to  other  offerors); 

•  Reveal  an  offeror’s  solution,  technology,  or  intellectual  property  to  another  offeror; 

•  Reveal  an  offeror’s  price  without  that  offeror’s  permission; 

•  Reveal  the  name  of  individuals  providing  past  performance  information;  or 

•  Knowingly  furnish  source  selection  information. 

Award  Without  Discussions 

Before  issuing  the  solicitation,  you  must  decide  whether  or  not  you  intend  to  award  without 
discussions  and  communicate  your  intent  in  the  solicitation.  In  making  this  decision,  consider 
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whether  or  not  you  are  likely  to  obtain  best  value  without  discussions.  An  award  without 
discussions  is  most  likely  to  result  in  best  value  when  requirements  are  clear,  commodities  are 
known  or  stable,  and  the  marketplace  is  extremely  competitive. 

If  your  solicitation  advised  offerors  of  intent  to  award  without  discussions,  you  may  still  hold 
discussions,  if  appropriate,  provided  you  document  the  file  as  to  why  discussions  are  necessary. 

If,  after  proposal  evaluation,  it  is  clear  that  the  cost  of  conducting  discussions  would  more  than 
offset  the  potentially  lower  prices  or  increased  functionality  resulting  from  discussions,  then  it 
may  be  appropriate  to  award  on  initial  proposals. 

Requesting  Clarifications  When  Awarding  Without  Discussions 

The  most  limited  exchanges  are  clarifications  that  occur  if  award  will  be  made  without 
discussions.  Under  these  circumstances,  we  may  give  offerors  the  opportunity  to  clarify  certain 
aspects  of  their  proposals  such  as  questions  about  the  relevancy  of  their  past  performance  or 
adverse  past  performance  information  on  which  an  offeror  hasn’t  yet  had  an  opportunity  to 
comment.  These  exchanges  may  be  used  to  resolve  minor  irregularities,  informalities,  or  clerical 
errors.  Such  clarifications  provide  minor  explanations  but  do  not  revise  or  modify  the  proposal, 
except  to  the  extent  that  correction  of  apparent  clerical  mistakes  results  in  a  modification. 

Holding  Communications 

Before  making  a  competitive  range  decision,  you  may  need  to  hold  communications  with  some 
offerors  to  determine  whether  or  not  to  include  a  proposal  in  the  competitive  range.  This  is  like 
fact-finding.  The  objective  of  these  precompetitive  range  exchanges  is  to  help  evaluators 
understand  and  evaluate  the  proposal. 


Communications  may  be  held 
to  help  evaluators  understand 
gray  areas  in  the  proposal 


Communications  must  be  held  with  any  offeror  who  will  be  excluded  from  the  competitive  range 
because  of  their  adverse  past  performance  information.  Otherwise,  you  may  hold 
communications  only  with  those  offerors  who  are  neither  clearly  in  nor  clearly  out  of  the 
competitive  range.  If  you  know  that  you  will  include  an  offeror  in  the  competitive  range,  then 
wait  until  you  open  discussions  to  address  your  concerns. 

Offerors  should  ensure  that  initial  proposals  are  as  clear  and  complete  as  possible.  When  holding 
communications,  ask  only  those  questions  necessary  to  understand  the  proposal  and  make  the 
competitive  range  determination.  You  may  use  communications  to  solicit  information  that  will 
clear  up  gray  areas,  such  as  perceived  deficiencies,  omissions,  and  errors,  or  questions  about  an 
offeror’s  capability  or  preaward  survey.  During  communications,  you  must  give  offerors  an 
opportunity  to  address  any  adverse  past  performance  information  to  which  the  offeror  has  not 
previously  had  an  opportunity  to  comment.  This  ensures  that  offerors  are  not  excluded  from  the 
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competitive  range  on  the  basis  of  incorrect  past  performance  information  that  they  had  not  had  a 
prior  opportunity  to  address. 

Information  obtained  during  communications,  however,  may  not  be  used  to  revise  a  proposal, 
correct  any  deficiencies  or  material  omissions,  or  change  any  technical  or  cost  elements  of  a 
proposal,  except  for  correction  of  mistakes. 


Communications  do  not 
permit  proposal  revisions 


Once  you  have  enough  information  to  decide  how  the  proposal  should  be  rated,  (e.g.,  decided 
whether  a  potential  deficiency  is,  indeed,  a  deficiency),  then  STOP.  Never  accept  a  revision 
before  opening  discussions. 


Establishing  the  Competitive  Range 

The  competitive  range  consists  of  all  the  most  highly  rated  proposals,  unless  it  is  further  reduced 
for  efficiency.  Establishing  the  competitive  range  results  in  greater  efficiency  by  limiting  the 
number  of  offerors  with  whom  the  Government  must  hold  discussions  to  the  finalists  or  leading 
contenders  for  contract  award.  However,  failure  to  properly  establish  a  competitive  range  can 
result  in  higher  costs  because  of  protests  or  eliminating  potentially  competitive  offerors.  When 
establishing  the  competitive  range,  consider  the  following  points: 

•  Determine  the  competitive  range  only  after  an  initial  evaluation  of  each  proposal  in 
accordance  with  all  cost  and  noncost  factors  in  the  solicitation. 

•  Limit  the  competitive  range  to  all  of  the  most  highly  rated  proposals,  considering  the 
initial  evaluation  of  both  cost  and  noncost  factors.  Predetermined  “cut-off’  ratings 
cannot  be  used  to  exclude  a  proposal  from  the  competitive  range. 

•  If  there  are  very  few  highly  rated  proposals,  you  may  want  to  include  all  of  them  in 
the  competitive  range. 

•  If  there  are  too  many  highly  rated  proposals  to  evaluate  efficiently,  you  may  limit  the 
competitive  range  farther,  provided  you  notified  offerors  of  your  intent  to  do  so  in 
the  solicitation. 

•  It  may  not  always  be  necessary  or  even  advisable  to  further  narrow  the  competitive 
range  for  efficiency.  You  must  determine  what  constitutes  an  efficient  competitive 
range  for  each  acquisition.  When  faced  with  the  need  to  restrict  the  size  of  the 
competitive  range,  you  should  consider  factors  such  as  the  expected  dollar  value  of 
the  award;  the  complexity  of  the  acquisition  and  solutions  proposed;  or  the  extent  of 
available  resources  and  other  relevant  matters  consistent  with  the  need  to  obtain  the 
best  value. 


148 


•  When  further  reducing  the  competitive  range  for  efficiency,  select  from  among  the 
most  highly  rated  proposals,  the  largest  number  that  will  still  permit  an  efficient 
competition. 

•  The  contracting  officer  determines  the  competitive  range.  In  the  case  of  more 
complex  source  selections,  the  determination  is  made  with  the  approval  of  the  source 
selection  authority. 

•  Document  the  competitive  range  determination  and  the  supporting  rationale  in  the 
contract  file. 

•  Maintain  an  efficient  competitive  range  that  doesn’t  waste  resources  for  either  side. 
The  competitive  range  should  be  continually  reassessed  as  discussions  and 
eyaluations  continue.  The  contracting  officer  should  remove  from  the  competitive 
range  any  proposal  that,  during  or  after  discussions,  is  no  longer  considered  to  be  a 
leading  contender  for  award.  This  allows  offerors  who  are  not  likely  to  be  selected 
for  award  to  shift  their  bid  and  proposal  costs  to  competitions  where  they  have  a 
better  chance  for  success.  The  objective  is  an  efficient  competitive  range  that 
doesn’t  string  offerors  along  wasting  their  time  and  money  and  your  resources. 


Maintain  an  efficient  competitive 
range  that  doesn’t  waste  resources 


•  For  proposals  excluded  from  the  competitive  range,  the  contracting  officer  shall 
promptly  notify  unsuccessful  offerors,  in  writing,  of  their  exclusion.  Upon  request, 
you  will  also  have  to  provide  a  debriefing  that  explains  the  basis  for  your  decision. 
See  Appendix  F  for  more  information  on  debriefings. 

Conducting  Discussions 

The  most  detailed  and  extensive  exchanges  are  negotiations  that  are  held  after  establishment  of 
the  competitive  range.  These  exchanges  are  known  as  discussions.  Unless  the  solicitation 
informs  offerors  that  award  may  be  made  without  discussions,  you  must  hold  meaningful 
discussions  with  each  offeror  in  the  competitive  range. 

The  primary  purpose  of  discussions  is  to  maximize  our  ability  to  get  the  best  value. 


Discussions  maximize  our 
ability  to  get  the  best  value 


During  discussions,  our  objective  should  be  to  reach  complete  agreement  between  and 
understanding  by  the  Government  and  the  offeror  regarding  all  the  basic  requirements  in  the 
solicitation.  In  essence,  obtaining  a  contract  that  demonstrates  the  greatest  promise  of  meeting 
the  solicitation’s  requirements  and  no  surprises  after  award  is  the  goal  of  both  the  Government 
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and  the  offeror.  While  the  content  of  discussions  is  a  matter  primarily  within  the  discretion  of 
the  contracting  officer,  discussions  must  meet  fundamental  requirements  to  be  meaningful  and 
fair. 


Discussions  must  be  meaningful  and  fair 


Ensure  discussions  are  meaningful  by  identifying  to  the 
offeror  all  evaluated  deficiencies,  significant  weaknesses, 
and  other  proposal  aspects  that  could  be  altered  or  explained 
to  enhance  materially  an  offeror’s  award  potential 

Confine  and  tailor  your  discussions  exclusively  to  each  offeror’s  proposal  relative  to  the 
solicitation  requirements  and  evaluation  factors  and  subfactors.  Identify  those  things  in  the 
proposal  that  could  clearly  limit  an  offeror’s  award  potential.  Seeking  the  advice  of  legal 
counsel  during  the  discussion  process  may  help  avoid  protests. 

You  can  facilitate  meaningful  discussions  by  addressing  the  following  as  a  minimum: 

•  Deficiencies  -  A  material  failure  to  meet  a  requirement.  It  is  a  deficiency  whenever 
the  offeror  specifically  says  a  requirement  cannot  or  will  not  be  met,  offers  an 
approach  that  clearly  doesn’t  meet  a  requirement,  or  submits  a  proposal  that  contains 
a  combination  of  significant  weaknesses. 

•  Significant  Weaknesses  —  Include  noncost  and  cost  weaknesses  that  appreciably 
increase  the  risk  of  unsuccessful  contract  performance.  It  is  a  weakness  whenever 
the  proposal  has  a  flaw  important  enough  to  cause  a  factor  to  be  rated  marginal  or 
poor,  or  the  probability  of  meeting  a  requirement  to  be  high  risk  or  moderate  to  high 
risk.  This  includes  even  relatively  minor  weaknesses  if  their  cumulative  impact  is 
significant.  For  example,  if  an  approach  affects  several  areas  of  the  evaluation,  but 
makes  no  individual  factor  rating  marginal  or  poor,  you  should  include  it  in 
discussions  if  the  cumulative  impact  is  significant  enough  to  impact  the  overall 
rating. 

•  Past  Performance  Information  -  Include  any  concern  about  an  offeror’s  past 
performance,  including  relevancy  and  any  adverse  past  performance  information  on 
which  the  offeror  has  not  previously  had  an  opportunity  to  comment. 

•  Uncertainties  or  apparent  mistakes  -  Include  any  suspected  errors,  any  significant 
omissions,  and  any  uncertainties  necessary  to  understand  what  is  being  offered. 
However,  perfect  knowledge  isn’t  necessary.  We  tend  to  spend  too  much  of  our  time 
and  effort  and  that  of  the  offeror,  chasing  information  that  has  no  real  bearing  on  the 
evaluation.  If  we  need  it  to  draw  a  conclusion,  then  we  should  ask  for  it. 

Identify  deficiencies  and  significant  weaknesses  in  terms  of  a  clear  declarative  statement. 
Advising  offerors  of  strengths  in  their  proposals  can  also  give  offerors  insight  into  areas  to 
consider  in  making  tradeoffs  to  correct  deficiencies  or  weaknesses. 
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Obtaining  Proposal  Revisions 


Confirm  all  information  obtained  through  discussions  by  requesting  or  allowing  proposal 
revisions,  as  appropriate,  from  all  offerors  in  the  competitive  range  still  eligible  for  selection. 
Proposals  are  rarely  alike,  nor  are  the  depth  and  range  of  discussions,  therefore,  tailor  the  number 
and  content  of  revisions  to  each  offeror’s  proposal.  Ask  offerors  to  submit  written  changes  to 
their  proposals  resulting  from  discussions  before  requesting  final  proposal  revisions,  particularly 
if  a  number  of  significant  issues  need  resolution.  This  allows  further  discussions,  if  necessary 
before  the  final  cutoff  date. 


Tailor  the  number  and  content  of 
revisions  to  each  offeror’s  proposal 


After  you  have  received  responses  to  all  issues  raised  to  the  offerors  during  discussions,  you 
must  reevaluate  the  proposals.  Any  factor  impacted  by  the  responses  must  be  rated  again  in  the 
same  manner  as  in  the  initial  evaluation.  Ensure  that  all  issues  are  resolved  or  understood  by 
each  offeror  and  the  government  prior  to  concluding  discussions. 

At  the  conclusion  of  discussions,  you  must  give  all  offerors  remaining  in  the  competitive  range 
an  opportunity  to  improve  their  proposal  by  submitting  a  final  proposal  revision  within  a 
common  cutoff  date  and  time.  If,  after  receipt  of  final  revised  proposals  it  becomes  necessary  to 
subsequently  clarify  minor  irregularities,  you  can,  without  any  additional  request  for  final 
proposal  revisions  from  all  offerors.  However,  if  you  need  to  negotiate  further,  a  second  final 
revision  opportunity  must  be  extended  to  all  offerors. 
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Selection  Decision 

Ensure  the  selection  decision: 

•  Is  based  on  a  comparative  analysis  of  the  proposals; 

•  Is  consistent  with  stated  evaluation  factors  and  subfactors;  and 

•  When  tradeoffs  are  permitted,  consider  whether  or  not  perceived  benefits  are 
worth  any  price  premium. 

•  Make  the  decision  on  a  rational  basis  and  set  it  forth  in  an  independent, 
stand-alone  defensible  document. 

Consistent  with  the  solicitation,  after  the  team  has  completed  the  evaluation  of  the  individual 
proposals,  the  source  selection  authority  compares  competing  proposals  to  each  other. 

When  using  the  lowest  price  technically  acceptable  process,  the  source  selection  authority 
compares  proposals  on  the  basis  of  cost  or  price  alone  and  selects  the  offeror  with  the  lowest 
evaluated  cost/price  meeting  the  acceptability  requirements  for  all  factors  and  subfactors. 

When  using  the  tradeoff  process,  the  source  selection  authority  compares  proposals  on  the  basis 
of  cost/price,  technical  or  other  noncost  ratings,  and  how  its  strengths,  weaknesses,  and  risks  will 
impact  the  specific  objectives  of  the  acquisition.  The  source  selection  authority  may  request  the 
evaluators  to  conduct  comparative  analyses  of  proposals  and  make  a  recommendation 
concerning  the  source  selection.  The  source  selection  authority  will  use  all  the  information  on 
the  proposals  and  evaluation  to  make  an  independent  judgement  of  the  best  value. 

Consistent  with  the  solicitation,  the  possible  outcomes  of  this  comparison  are: 

•  The  proposal  with  the  superior  noncost  merit  is  the  lowest  cost/price  proposal.  In 
this  case  award  should  be  made  to  the  offeror  submitting  the  proposal  with  the 
lowest  evaluated  price  or  cost. 

•  The  proposals  may  be  determined  to  be  essentially  equal  in  terms  of  noncost  factors. 
In  this  case  also,  award  should  be  made  to  the  offeror  submitting  the  proposal  with 
the  lowest  evaluated  price  or  cost. 

•  When  the  proposal  with  the  lowest  evaluated  price  or  cost  is  other  than  the 
proposal(s)  with  higher  noncost  merit,  the  source  selection  authority  must  perform  a 
cost/technical  tradeoff  analysis  to  decide  whether  the  technical  superiority  of  the 
other  proposal(s)  warrants  payment  of  the  additional  price  or  cost. 

Making  the  Cost/Technical  Tradeoff  Analysis 

Ratings  are  merely  guides  for  decision  making.  The  source  selection  authority  is  responsible  for 
independently  determining  whether  noncost  advantages  are  worth  the  cost/price  that  might  be 
associated  with  a  higher  rated  proposal.  The  decisive  element  is  not  the  difference  in  ratings,  but 
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the  source  selection  authority’s  rational  judgement  of  the  significance  of  that  difference,  based 
on  an  integrated  comparative  assessment  of  proposals. 


There  is  no  magic  formula  for 
making  the  cost/technical  tradeoff 


When  making  the  cost/technical  tradeoff  leading  to  the  selection  decision,  there  is  no  “magic” 
formula.  The  cost/technical  tradeoff  and  the  source  selection  decision,  which  must  be  consistent 
with  the  solicitation,  require  that  the  source  selection  authority  exercise  reasonable  business 
judgment  in  selecting  the  offeror  for  contract  award.  The  information  considered  should  include 
an  analysis  of  the  following: 

•  The  proposals’  total  evaluated  price  or  cost. 

•  The  significance  of  the  differences  in  the  noncost  ratings  as  indicated  by  each 
proposal’s  strengths,  weaknesses,  and  risks.  The  strengths,  weaknesses,  and  risks  for 
each  factor  must  be  considered  in  light  of  the  relative  importance  of  each  factor 
stated  in  the  solicitation. 

In  performing  a  tradeoff,  consider  following  steps  such  as  these  to  arrive  at  a  rationale  decision 
that  can  be  well  documented: 

•  Compare  the  proposal  differences  that  surfaced  during  your  evaluations; 

•  Define  these  differences  and  analyze  their  impact  on  performance  objectives; 

•  Make  paired  comparisons,  comparing  each  proposal  to  each  of  the  others; 

•  Assess  the  best  mix  of  cost  and  noncost  benefits  and  determine  whether  the  strengths 
of  higher  rated  proposals  are  worth  the  price  premium. 


A  price  premium  must  be  justified 
regardless  of  the  superiority  of  the  rating 


It  is  essential  to  document  cost/technical  tradeoff  judgments  with  detailed  narrative  explaining 
the  relevant  facts  and  supporting  rationale.  Mere  statements  of  conclusion  based  on  ratings  or 
scores  alone  are  not  acceptable.  The  cost/technical  tradeoff  documentation  must  explicitly 
justify  a  price  premium  regardless  of  the  superiority  of  the  selected  proposal’s  technical  or 
noncost  rating.  This  justification  is  required  even  when  the  solicitation  indicates  that  noncost 
factors  are  more  important  than  cost/price.  The  justification  must  clearly  state  what  benefits  or 
advantages  the  Government  is  getting  for  the  added  cost/price  and  why  it  is  in  the  Government’s 
interest  to  expend  the  additional  funds. 
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Where  it  is  determined  that  the  noncost  benefits  offered  by  the  higher  priced,  technically  superior 
offeror  are  not  worth  the  price  premium,  an  explicit  justification  is  also  necessary.  In  this  case, 
the  documentation  must  clearly  show  why  it  is  reasonable  in  light  of  the  significance  of  the 
differences  to  pay  less  money  for  a  proposal  of  lesser  technical  merit. 

To  determine  which  proposal  provides  the  best  value,  the  source  selection  authority  must  analyze 
the  differences  between  competing  proposals.  This  analysis  must  be  based  on  the  facts  and 
circumstances  of  each  acquisition  and  must  be  consistent  with  the  solicitation. 

This  analysis  ensures  a  disciplined  and  documented  process  for  an  integrated  comparison  of 
proposals  and  a  rational  basis  for  the  source  selection  authority’s  ultimate  decision. 

Documenting  the  Proposal  Comparison 

Documentation  explaining  the  final  results  of  the  evaluation  should  be  prepared  for  the  source 
selection  authority  to  use  in  making  the  selection  decision.  This  documentation  should  include 
the  technical  and/or  past  performance  evaluation  results,  the  cost/price  evaluation,  and  the 
comparative  value  analysis,  if  applicable,  for  each  proposal  in  the  competitive  range.  The 
documentation  should  also  include  other  considerations  such  as  the  results  of  negotiations. 

For  more  complex  source  selections,  this  is  accomplished  by  means  of  a  formal  report  that  is 
provided  to  the  source  selection  authority.  For  less  complex  source  selections,  the 
documentation  may  be  included  as  part  of  the  Price  Negotiation  Memorandum.  It  should  be 
simple  but  concise  and  should  cross-reference  rather  than  repeat  information  in  existing 
documents  as  much  as  possible  (e.g.,  the  source  selection  plan,  evaluation  team  consensus 
report).  The  analysis  and  comparisons  in  this  documentation  should  be  used  as  an  aid  to  the 
source  selection  authority’s  judgment  --  not  as  a  substitute  for  judgment. 

The  documentation  may  contain: 

•  Introductory  information  such  as: 

•  Data  about  the  source  selection  plan. 

•  The  basis  for  award  and  evaluation  factors  and  subfactors. 

•  Participants  in  the  evaluation  process. 

•  Solicitation  requirements. 

•  The  number  of  offerors  solicited. 

•  The  offerors  who  responded  and  those  in  the  competitive  range. 

•  A  summary  of  each  proposal  within  the  competitive  range 

Summary  matrix  shows  how  evaluation  results  for  each  offeror  are  integrated  at  the 
factor  level  for  use  in  documenting  the  proposal  comparison.  All  ratings  must  be 
supported  with  narrative  that  describes  the  proposal’s  strengths,  weaknesses,  and  risks. 

“  FACTORS 
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TECHNICAL  MERIT 

Summarizes  assessment  of  the  offeror’s 
proposal  as  measured  against  the  technical 
subfactors 
Example:  “Good” 

PROPOSAL  RISK 

Summarizes  assessment  of  the  weaknesses 
and  risks  associated  with  the  offeror’s 
proposed  approach  derived  from  the 
technical  evaluation 
Example:  Low  Risk 

PERFORMANCE  RISK 


COST/PRICE 

Reflects  the  total  proposed  cost  or  price. 
Where  cost  realism  is  evaluated,  the  cost 
also  reflects  the  probable  cost  resulting 
from  any  adjustments  made  for  cost 
realism. 

Example:  Proposed  Cost  $XXX 
Most  P  Most  Probable  Cost  $XXX 


Blueprint:  Individual  Offeror  Evaluation  Results 

•  Comparative  analyses  of  both  cost  and  noncost  factors  of  the  proposals  within  the 
competitive  range.  The  factors  and  subfactors  evaluated  should  be  discussed,  first 
individually  and  then  comparatively.  The  comparative  cost  analysis  should  explain 
the  reasonableness,  realism,  and  rationale  of  each  offeror’s  price  or  cost  proposal. 
Each  proposal’s  major  strengths,  weaknesses,  risks,  as  well  as  the  details  and  results 
of  the  tradeoff  analysis  should  be  included. 

Matrix  shows  summary  comparative  evaluation  results  for  all  offerors  within  the  competitive 
range.  The  supporting  comparative  analysis  must  document  the  integrated  assessment  of  the 
technical  (merit  and  risk),  performance  risk,  and  evaluated  cost  of  the  proposals  relative  to  the 
factors  and  subfactors  and  to  each  other. 

TECHNICAL  PERFORMANCE  EVALUATED  COST 


Summarizes  assessment  of  the  offeror’s 
demonstrated  performance  on  recent, 
relevant  contracts  relative  to  the  past 
performance  subfactors. 

Example:  Moderate  Risk 
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OFFEROR 

TECHNICAL  MERIT 

PROPOSAL  RISK 

RISK 

(Most  Probable  Cost) 

A 

Excellent 

Low 

Low 

$171,503,971 

B 

Excellent 

Moderate 

Moderate 

$134,983,305 

C 

Good 

Low 

Moderate 

$120,976,836 

D 

Excellent 

High 

Moderate 

$150,840,308 

E 

Satisfactory 

Moderate 

Low 

$115,751,933 

Blueprint:  Summary  Comparison  Matrix 

•  A  discussion  of  the  overall  impact  of  significant  risks  associated  with  each  proposal 
within  the  competitive  range.  This  discussion  may  address,  for  example: 

•  Technical  risks  inherent  in  the  offeror’s  proposed  approach. 

•  Degree  of  confidence  in  the  realism  of  the  offeror’s  cost  or  price  proposal  taking  into 
consideration  technical  and  schedule  risk. 

•  Production  risks  relating  to  new  technologies  and  overall  production  competence. 

•  Performance  risks  relative  to  the  offeror’s  record  of  recent  and  relevant  past 
performance. 

•  A  summary  of  the  comparative  analyses,  expressed  in  brief  statements,  of  the  issues 
considered  significant  to  the  source  selection  authority’s  decision.  If  requested  by 
the  source  selection  authority,  a  selection  recommendation  would  be  included. 

Documenting  the  Selection  Decision  and  Awarding  the  Contract 
Source  Selection  Decision  Document  for  (specify  product/service  &  RFP  #) 

1.  Decision  Statement. 

Example:  As  Source  Selection  Authority  for  this  acquisition,  I  have  determined  that  the  XYZ 
product/service  proposed  by  Offeror  C  provides  the  best  overall  value  to  satisfy  Army  needs. 
This  selection  was  made  based  upon  the  factors  and  subfactors  established  in  the  solicitation  and 
my  integrated  assessment  and  comparison  of  the  strengths,  weaknesses,  and  risks  of  the 
proposals  submitted  in  response  to  the  solicitation.  This  memorandum  documents  the  basis  for 
my  decision. 

2.  Brief  description  of  the  product/service  called  for  in  the  solicitation. 

3.  Brief  description  of  the  Basis  for  Award  including  the  major  factors  against  which 
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proposals  were  measured  and  their  relative  order  of  importance. 

4.  A  list  of  offerors  in  the  competitive  range. 

5.  Rationale  for  business  judgments  and  tradeoffs.  Include  the  following: 

•  Succinctly  compare  each  proposal  to  each  of  the  others,  focusing  on  key  proposal  differences 
(strengths,  weaknesses,  and  risks)  that  surfaced  in  the  evaluation  and  their  impact  on  the 
acquisition  objectives. 

•  Clearly  explain  specific  tradeoffs  that  led  to  the  decision. 

•  Clearly  explain  the  specific  benefits  of  technically  superior  offeror(s)  and  why  they  are  or  are 
not  significant  enough  to  warrant  any  additional  cost. 

6.  Summary. 

Example:  In  summary,  based  on  my  integrated  assessment  of  all  proposals  in  accordance  with 
the  specified  evaluation  factors  and  subfactors,  it  is  my  decision  that  Offeror  C’s  proposal  offers 
the  best  overall  value. 

Signature 

Source  Selection  Authority 


Blueprint:  Typical  Source  Selection  Decision  Document 

Documentation  setting  forth  the  decision  rationale  must  be  prepared  to  support  the  source 
selection  authority’s  decision.  The  selection  statement  must  be  a  stand-alone  document  that 
succinctly  and  accurately  provides  rationale  for  the  selection.  It  should  explain  how  the 
successful  proposal  measured  up  against  other  offerors  based  on  the  evaluation  factors  and 
subfactors  in  the  solicitation.  It  should  also  explain  the  tradeoff  judgments,  including  benefits 
associated  with  additional  cost. 

This  document  becomes  part  of  the  official  contract  file  and  can  even  be  released,  provided  that 
any  information  exempt  under  the  Freedom  of  Information  Act  (FOIA)  is  not  released.  This  can 
ease  the  debriefing  process  by  showing  offerors  who  request  a  debriefing  the  rationale  and  logic 
used  by  the  source  selection  authority.  After  the  source  selection  authority  has  signed  the 
selection  decision  document,  the  contracting  officer  may  execute  and  distribute  the  contract. 
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Notification  and  Debriefing  of  Unsuccessful  Offerors 

When  a  contract  is  awarded  as  the  result  of  a  source  selection,  unsuccessful  offerors  must  be 
debriefed  and  furnished  the  basis  for  the  selection  decision  and  contract  award  upon  their  written 
request.  In  addition,  offerors  excluded  from  the  competitive  range  or  otherwise  excluded  from 
the  competition  before  award  may  request  either  a  preaward  or  postaward  debriefing.  A 
debriefing  may  also  be  provided  to  the  successful  offeror. 

Debrief  offerors  promptly,  at  their  request,  as  to  the  basis 
for  the  selection  decision.  Candidly  explain  the  results  of 
the  Government’s  evaluation  of  their  proposal  without 
making  any  point-by-point  comparisons  with  the  content 
of  other  proposals. 

It  is  extremely  important  to  promptly  notify  and  debrief  an  offeror.  Since  each  offeror  puts 
considerable  resources  into  preparing  and  submitting  a  proposal,  fairness  dictates  that  you 
explain  why  a  proposal  was  unsuccessful.  Early  notification  will  also  permit  unsuccessful 
offerors  to  release  the  resources  that  would  have  been  devoted  to  the  contract  effort  so  they  can 
be  used  on  other  work.  It  is  also  in  the  Government’s  best  interest  to  fully  inform  the  offeror  of 
the  proposal’s  shortcomings  so  that  the  same  mistakes  are  not  repeated  in  future  acquisitions. 
These  actions  reduce  the  cost  of  the  competitive  process  and  encourage  the  offeror  to  view  the 
Government  marketplace  as  a  worthwhile  area  to  invest  its  resources,  thereby  increasing 
competition.  See  Appendix  F  for  guidelines  on  conducting  debriefings. 
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Top  Ten  Messages 

1.  Use  a  source  selection  approach  commensurate  with  the  acquisition ’s  evaluation  needs. 

•  Don  V  make  source  selection  more  complicated  and  expensive  than  necessary. 

2.  Invest  in  command  or  program  resources  needed  for  a  competent  and  well  documented 
best  value  source  selection. 

•  Include  the  source  selection  authority  as  an  active  participant  —  this  is  not  a 
“figure  head”  position. 

•  Train  evaluators  in  best  practices. 

3.  Understand  the  importance  of  planning. 

•  Think  through  the  entire  process. 

•  Prepare  a  source  selection  plan  before  the  solicitation. 

4.  Structure  the  solicitation  to  communicate  effectively  to  potential  offerors: 

•  Government  requirements  and  mission  objectives. 

•  The  information  needed  to  evaluate  the  proposal 

•  The  ground  rules  the  government  will  use  to  select  the  best  value  proposal. 

•  Evaluation  factors  and  subfactors  and  their  relative  importance. 

5.  State  requirements  functionally  to  the  maximum  extent  possible. 

•  Limit  use  of  design  requirements. 

6.  Document  strengths,  weaknesses,  risks  and  associated  value  of  proposals  to  support  the 
cost/technical  tradeoff. 

•  Recognize  that  numerical  rating  systems  can  imply  false  precision  and  limit 
flexibility  in  the  decision  making  process. 

7.  Justify  a  price  premium  with  cost/technical  tradeoff  documentation  regardless  of  the 
selected  proposal’s  technical  superiority. 

8.  Ensure  that  the  source  selection  decision  is  consistent  with  the  solicitation. 

9.  Do  a  proper  and  timely  debriefing. 

1 0.  Document  lessons  learned. 


The  proponent  of  this  pamphlet  is  the  United  States  Army 
Materiel  Command  (AMC).  Please  send  comments  and 
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suggested  improvements  to  U.S.  Army  Materiel  Command, 
Attn:  AMCRDA-AP,  5001  Eisenhower  Avenue,  Alexandria, 
VA  22333-0001  or  submit  your  comments  via  the  Internet 
at  the  following  address:  http://www.army- 
acquisition.net/amcweb/fr_ssll.  htm. 


For  the  Commander: 
Official: 


James  M.  Link 
Major  General  USA 
Chief  of  Staff 

Leroy  Tillery 

Chief,  Printing  and  Publications  Branch 
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Definitions 


Competitive  Range.  All  proposals  that  the  contracting  officer  determines  to  be  the  most  highly 
rated  based  on  the  cost  and  noncost  evaluation  factors  stated  in  the  solicitation. 

Design  Specification.  A  specification  that  establishes  precise  measurement,  tolerances, 

materials,  in  process  and  finished  product  tests,  quality  control,  inspection  requirements, 
and  other  specific  details  of  the  deliverable. 

Evaluation  Factors  and  Subfactors.  Descriptions  of  those  aspects  of  a  proposal  that  will  be 
evaluated  quantitatively  or  qualitatively  to  assess  which  proposal  can  best  meet  the 
Government’s  requirements  as  described  in  the  solicitation. 

Evaluation  Standards.  A  baseline  level  of  merit  or  acceptability  used  for  measuring  how  well 
or  whether  an  offeror’s  response  meets  the  solicitation’s  requirements  with  respect  to  an 
evaluation  factor  or  subfactor. 

Functional  Specification.  A  specification  that  describes  the  deliverable  in  terms  of  form,  fit, 
and  function  and  performance  characteristics  to  satisfy  the  intended  use. 

Performance  Specification.  A  specification  that  states  requirements  in  terms  of  the  required 
results  and  provides  criteria  for  verifying  compliance,  without  stating  methods  for 
achieving  results.  It  defines  the  functional  requirements,  the  operational  environment,  and 
the  interface  and  interchangeability  requirements. 

Rating.  The  application  of  a  scale  of  words,  colors,  or  numbers,  used  in  conjunction  with 
narrative,  to  denote  the  degree  to  which  the  proposal  has  met  the  standard  for  a  noncost 
factor. 

Source  Selection/Evaluation  Plan.  The  document  that  explains  how  proposals  are  to  be 

solicited  and  evaluated  to  make  the  selection  decision.  It  is  the  Government’s  plan  for  how 
it  intends  to  acquire  its  needs. 

Source  Selection  Process.  The  process  of  soliciting  and  evaluating  proposals  for  award  in  a 
competitively  negotiated  environment. 

Specification.  A  description  of  the  technical  requirements  for  a  material,  product,  or  service  to 
be  provided  under  a  contract  that  includes  the  criteria  for  determining  whether  the 
requirements  are  met. 

Statement  of  Work  (SOW).  The  complete  description  of  work  to  be  performed  under  the 
contract,  encompassing  all  specifications  and  standards  established  or  referenced  in  the 
contract. 

Work  Breakdown  Structure.  A  basic  framework,  similar  to  a  table  of  contents,  which  outlines, 
divides,  and  subdivides  to  successively  lower  levels,  the  government’s  requirements  as  set 
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forth  in  the  SOW.  Permits  a  logical  arrangement  of  the  elements  of  the  SOW  and  a  tracing 
of  work  effort  and  costs  proposed  by  each  offeror  under  each  of  the  elements. 
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Guidelines  for  Evaluating  Past  Performance  in  Source  Selection 


Overview 


This  appendix  is  designed  to  help  you  participate  in  the  evaluation  of  past  performance  when 
using  the  tradeoff  process  in  source  selection.  It  should  always  be  used  in  conjunction  with  the 
basic  best  value  guide. 


Introduction 


An  offeror’s  performance  record  is  an  important  consideration  in  government  source  selections. 
In  our  private  lives,  we  make  source  selections  every  day.  This  can  be  as  mundane  as  selecting 
the  brand  of  toothpaste  we  use  or  one  that  represents  a  much  greater  portion  of  our  earnings  such 
as  a  new  car.  If  we  can  consider  how  a  product  worked  (or  didn’t  work)  for  us  in  the  past,  get 
advice  from  our  friends,  read  Consumer  Reports,  and  check  the  contractor  out  with  the  Better 
Business  Bureau,  wouldn’t  it  make  sense  to  give  the  same  credence  to  past  performance  when 
spending  taxpayer  dollars? 

Using  the  contractor’s  past  performance  as  a  significant  evaluation  factor  that  will  be  traded  off 
with  cost/price  and  other  noncost  factors  will  benefit  us  in  at  least  two  ways. 

First,  it  is  virtually  impossible  for  an  offeror  to  convey,  in  the  proposal,  how  it  will  actually 
perform  the  requirement  and  what  the  actual  costs  will  be  (not  just  the  instant  contract  costs,  but 
the  total  life  cycle  costs  associated  with  obtaining  the  system,  supply,  or  service).  A  better  tool 
for  predicting  how  an  offeror  will  perform  in  the  future  is  to  examine  its  past  track  record  on 
contracts  for  similar  requirements  of  the  same  scope  and  complexity. 

Second,  it  permits  us  to  recognize  the  good  performer.  In  the  commercial  market,  poor 
performers  are  weeded  out  as  consumers  select  vendors  that  live  up  to  the  consumer’s 
expectations.  Those  contractors  that  provide  what  is  perceived  as  the  best  value  in  supplies  or 
services  are  the  ones  that  survive.  A  contractor  that  delivers  what  the  contract  requires  without 
extensive  follow-up  effort  on  our  part  is  clearly  delivering  better  value  than  a  contractor  that 
charges  the  same  price,  yet  needs  constant  surveillance  by  our  personnel  to  ensure  performance. 
It  also  shifts  the  emphasis  from  writing  the  best  proposal  to  performing  the  best  work  — 
performing  as  promised. 


Role  of  Past  Performance  in  Source  Selection 


Proposal  Risk  vs.  Performance  Risk 

It  is  important  to  differentiate  between  two  types  of  risk  usually  evaluated  in  a  source  selection. 

Proposal  risks  are  those  associated  with  an  offeror’s  proposed  approach  in  meeting  the 
requirements  of  the  solicitation.  Proposal  risk  summarizes  the  risk  derived  from  the  technical 
evaluation.  It  is  an  overall  assessment  driven  by  each  of  the  subfactors  within  the  technical 
factor. 
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Performance  risks  are  those  associated  with  an  offeror’s  likelihood  of  success  in  performing  the 
solicitation’s  requirements  as  indicated  by  that  offeror’s  record  of  current  or  past  performance. 
Performance  risk  is  assessed  separately  and  assigned  a  narrative  rating.  Both  proposal  and 
performance  risks  are,  in  accordance  with  the  basic  guide,  integrated  with  the  other  evaluation 
results  when  presented  to  the  source  selection  authority  for  a  final  decision. 


Performance  risks  reflect  an  offeror’s 
likelihood  of  success  based  on 
their  current  or  past  work  record 


Past  Performance  vs.  Responsibility  Determinations 

It  is  also  important  to  distinguish  comparative  past  performance  evaluations  used  in  the  tradeoff 
process  from  pass/fail  responsibility  determinations. 

Preaward  surveys  and  pass/fail  evaluations  in  the  low  price  technically  acceptable  process  help 
you  determine  whether  an  offeror  is  responsible.  Responsibility  is  a  broad  concept  that 
addresses  whether  an  offeror  has  the  capability  to  perform  a  particular  contract  based  upon  an 
analysis  of  many  areas  including  financial  resources,  operational  controls,  technical  skills,  „ 
quality  assurance,  and  past  performance.  These  surveys  and  evaluations  provide  a  yes/no, 
“pass/fail,”  or  “go/no-go”  answer  to  the  question,  “Can  the  offeror  do  the  work?”  to  help  you 
determine  whether  the  offeror  is  responsible. 

Unlike  a  pass/fail  responsibility  determination,  a  comparative  past  performance  evaluation 
conducted  using  the  tradeoff  process  is  a  very  specific  endeavor  that  seeks  to  identify  the  degree 
of  risk  associated  with  each  competing  offeror.  Rather  than  asking  whether  an  offeror  can  do 
the  work,  you  should  ask,  will  it  do  that  work  successfully?  In  short,  the  evaluation  describes  the 
degree  of  confidence  the  government  has  in  the  offeror’s  likelihood  of  success.  If  properly 
conducted,  the  comparative  past  performance  evaluation  and  the  responsibility  determination 
will  complement  each  other  and  provide  you  with  a  more  complete  picture  of  an  offeror  than 
either  one  could  by  itself. 


Past  Performance  vs.  Experience 

Another  important  issue  is  the  difference  between  contractors’  experience  and  their  past 
performance.  Experience  reflects  whether  contractors  have  performed  similar  work  before.  Past 
performance,  on  the  other  hand,  describes  how  well  contractors  performed  the  work.  In  other 
words,  how  well  did  they  execute  what  was  promised  in  the  proposal/contract.  Both  of  these 
areas  are  considered  when  making  a  responsibility  determination.  Either  past  performance  or 
experience  can  be  considered  as  source  selection  factors  or  subfactors,  where  they  can  either 
stand  alone  or  be  considered  under  performance  risk. 

If  experience  and  past  performance  are  separate  evaluation  factors  or  subfactors,  make  certain 
that  you  clearly  define  the  terms  in  the  solicitation.  This  will  help  you  avoid  the  potential  for 
double  counting  by  asking  for  the  same  information.  Do  not  confuse  evaluation  of  past 
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experience  with  evaluation  of  past  performance.  It  is  proper,  however,  to  distinguish  company 
experience  from  personnel  experience  and  evaluate  both. 

When  to  Evaluate  Past  Performance 

Past  performance  shall  be  included  as  an  evaluation  factor  in  competitively  negotiated 
acquisitions  unless  the  contracting  officer  determines  that  it  is  inappropriate  and  documents  the 
rationale. 


Planning  the  Past  Performance  Evaluation 
Forming  an  Evaluation  Group 

In  complex  acquisitions  it  may  be  necessary  to  form  a  formal  group  to  specifically  evaluate  past 
performance.  In  smaller  dollar  value  acquisitions  that  do  not  involve  complex  requirements,  the 
evaluation  may  be  accomplished  with  only  one  or  two  people  to  determine  that  same  risk.  This 
evaluation  group  may  operate  separately  from  the  proposal  evaluation  team  or  may  operate  as  a 
separate  subgroup  of  that  team.  Note,  however,  that  the  past  performance  evaluation  should  be 
conducted  independently  of  any  other  evaluation. 

Whether  you  choose  to  use  an  informal  or  formal  evaluation  group,  the  members  should  be 
experienced  and  trained.  We  will  focus  on  the  structure,  composition  and  evaluation  process  of  a 
formal  evaluation  group,  but  bear  in  mind  that  while  the  functions  of  informal  evaluations  are 
basically  the  same,  they  should  be  a  lot  less  time  consuming  and  less  involved. 

Objectives  of  the  Evaluation  Group 

The  evaluation  group  is  responsible  for  conducting  the  past  performance  evaluation  to  determine 
the  degree  of  risk  involved  in  accepting  an  offeror’s  promises  of  performance.  This  analysis 
results  in  a  performance  risk  assessment.  The  evaluation  group  documents  these  performance 
risk  assessments  and  identifies  strengths  and  weakness  in  each  offeror’s  past  performance. 

The  group  often  uses  the  offeror’s  proposal  as  the  starting  point  -  with  the  proposal  acting 
primarily  as  a  source  of  reference.  Group  members  then  use  independent  sources  of  information 
outside  of  the  offerors’  proposals  to  determine  how  well  those  offerors  performed  in  the  past. 

Evaluation  Group  Membership  and  Training 

The  membership  and  structure  of  your  evaluation  group  should  be  tailored  to  each  acquisition. 
Ideally,  the  membership  should  be  multifunctional  and  include  people  who  have  contracting, 
cost/price,  contract  administration,  supportability/logistics,  and  technical/program  management 
expertise,  as  appropriate.  Those  whom  you  select  should  also  have  prior  source  selection 
experience  and  be  capable  of  making  sound  and  impartial  judgments.  The  group’s  structure 
should  enhance  its  ability  to  independently  evaluate  performance  risk. 

As  a  best  practice,  it  is  important  that  contracting  organizations  set  up  their  own  training  program 
for  conducting  source  selections.  Individual  programs  should  include  tools  and  techniques  for 
obtaining  and  evaluating  contractor  performance  data. 
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The  heart  of  the  performance  risk  assessment  is  the  information  gathering  process.  Through 
questionnaires,  telephone  interviews,  and  site  visits,  and  by  tapping  existing  data  sources,  the 
group  can  obtain  a  detailed  and  useful  picture  of  an  offeror’s  past  performance.  Because  of  the 
importance  of  the  information  gathering  process,  it  is  absolutely  critical  that  group  members 
have  the  ability  to  conduct  meaningful  telephone  interviews.  They  should  also  be  able  to 
assimilate  data,  exercise  sound  judgment,  arrive  at  conclusions  that  make  common  sense,  and 
communicate  those  conclusions  effectively  both  orally  and  in  writing. 

The  size  of  the  group  should  reflect  the  number  of  offerors  expected  to  respond  to  the  solicitation 
as  well  as  the  nature  and  complexity  of  the  solicitation  requirements.  The  best  practice  is  to  limit 
the  size  of  the  group  to  as  small  a  number  as  is  realistic  for  the  specific  circumstances  of  the 
acquisition.  A  group  of  at  least  two  members  of  different  functional  disciplines  enhances 
opportunities  for  dialogue,  brainstorming,  and  in-depth  fact  finding. 

A  plan  for  evaluating  past  performance  should  be  developed  early  in  the  process  and  made  a  part 
of  the  source  selection  plan. 


What  Subfactors  Should  Be  Used? 

The  past  performance  subfactors,  if  any,  should  be  tailored  to  the  specific  circumstances  of  the 
acquisition,  but  need  not  mirror  those  of  the  proposal  evaluation.  In  most  cases  the  evaluation 
group  should  at  least  consider  the  offeror’s  record  for  on  time  delivery,  good  technical  quality, 
and  cost  control  to  determine  its  likelihood  of  success  in  performing  the  solicitation’s 
requirements. 

Some  suggested  subfactors  include: 

•  The  offeror’s  record  of  conformance  to  contract  specifications  and  standards  of  good 
workmanship.  This  may  include  consideration  of  quality  awards  or  certifications; 

•  The  offeror’s  effectiveness  in  containing  and  forecasting  costs  on  any  previously 
performed  cost  reimbursable  contracts; 

•  The  offeror’s  adherence  to  contract  schedules,  including  the  administrative  aspects  of 
performance; 

•  The  offeror’s  history  for  reasonable  and  cooperative  behavior,  commitment  to 
customer  satisfaction,  timely  award  and  management  of  subcontracts,  and  whether 
the  offeror  met  any  applicable  goals  for  subcontracting  with  small  entities. 

Data  Relevancy 

Relevancy  should  not  be  described  as  a  subfactor.  Relevancy  is  a  threshold  question  when 
considering  past  performance,  not  a  separate  element  of  past  performance.  Irrelevant  past 
performance  should  not  form  the  basis  of  a  performance  risk  assessment. 

Although  the  group  may  consider  data  available  from  any  sources,  its  main  sources  of 
information  are  often  the  references  cited  by  offerors  in  their  proposals.  Upon  receipt  of 
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proposals,  you  will  determine  which  of  the  offeror’s  past  contract  efforts  relate  to  the  solicitation 
requirements.  The  evaluation  group  should  screen  the  information  provided  for  each  of  the 
referenced  contracts  to  make  an  initial  determination  of  its  relevance  to  the  current  requirement. 
However,  the  source  selection  authority  is  responsible  for  the  final  determination  of  relevancy. 

Relevancy  is  defined  as  something  that  has  a  logical  connection  with  the  matter  under 
consideration.  Such  aspects  of  relevance  include  the  type  of  effort  (e.g.,  development, 
production,  repair),  and  the  type  of  requirement  (e.g.,  weapon  systems,  information  systems, 
engineering  services,  programmed  depot  maintenance).  The  objective  of  the  screening  is  to 
remove  from  consideration  those  contract  references  that  are  clearly  unrelated  to  the  type  of 
effort  sought.  Note  that  valuable  information  can  be  obtained  from  seemingly  unrelated  prior 
contracts  regarding  considerations  such  as  technical  capability,  management  responsiveness, 
proactive  process  improvements,  and  ability  to  handle  complex  technical  or  management 
requirements.  Other  members  of  the  source  selection  team  may  be  consulted  as  necessary  for 
assistance  in  determining  relevancy. 


Tailor  data  relevancy  and  recency 
to  the  specifics  of  each  acquisition 


In  some  cases,  previous  contracts  as  a  whole  may  be  similar  to  the  current  contract  while  in 
others  only  portions  of  previous  contracts  may  be  relevant.  For  example,  we  use  Ada  software 
language  in  many  different  government  systems.  If  a  solicitation  calls  for  the  development  of 
Ada  software  for  an  aircraft  system,  the  contractor  might  identify  a  previous  effort  where  it 
developed  Ada  software  for  a  satellite  terminal.  We  may  consider  that  previous  effort  to  be 
relevant  for  purposes  of  assessing  the  contractor’s  ability  to  develop  Ada  software  even  though 
the  underlying  system  is  different  from  the  current  requirement.  Another  example  is  the 
evaluation  of  the  contractor’s  management,  planning,  and  scheduling  of  subcontractors  on  a  past 
service  contract  for  a  current  production  requirement  calling  for  integration  skills. 

Note  that,  in  the  case  of  mergers  or  joint  ventures,  only  that  part  of  the  newly  formed  corporation 
identified  with  the  relevant  experience  should  be  evaluated. 

The  evaluation  group  should  consider  the  most  recent  data  available.  The  best  practice  is  to 
select  similar  efforts  that  are  either  still  in  progress  or  just  completed,  and  that  have  at  least  one 
year  of  performance  history.  While  the  actual  cut-off  time  should  be  determined  by  the 
contracting  officer  on  a  case-by-case  basis,  the  currency  of  the  information  requested  should  be 
determined  by  the  commodity  and  the  specific  circumstances  of  the  acquisition. 

How  Much  Weight  to  Give  Past  Performance 

Past  performance  should  be  given  sufficient  evaluation  weight  to  ensure  that  it  is  meaningfully 
considered  throughout  the  source  selection  process  and  will  be  a  valid  discriminator  among  the 
proposals  received. 


Weight  past  performance  sufficiently 
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to  ensure  it  is  a  valid  discriminator 


What  are  the  Rating  Categories? 

In  planning  the  acquisition,  the  evaluation  group  develops  a  rating  scheme  for  evaluating  past 
performance.  The  group  may  use  the  following  definitions  of  performance  risk  to  describe  the 
results  of  its  assessment: 

•  Poor/Very  High  Performance  Risk.  Based  on  the  offeror’s  performance  record, 
extreme  doubt  exists  that  the  offeror  will  successfully  perform  the  required  effort. 

•  Marginal/High  Performance  Risk.  Based  on  the  offeror’s  performance  record, 
substantial  doubt  exists  that  the  offeror  will  successfully  perform  the  required  effort. 

•  Adequate/Moderate  Performance  Risk.  Based  on  the  offeror’s  performance  record, 
some  doubt  exists  that  the  offeror  will  successfully  perform  the  required  effort. 

•  Good/Low  Performance  Risk.  Based  on  the  offeror’s  performance  record,  little 
doubt  exists  that  the  offeror  will  successfully  perform  the  required  effort. 

•  Excellent/Very  Low  Performance  Risk.  Based  on  the  offeror’s  performance  record, 
essentially  no  doubt  exists  that  the  offeror  will  successfully  perform  the  required 
effort. 

•  Unknown  Performance  Risk.  No  performance  record  identifiable.  See  “How  to 
Evaluate  No  Past  Performance.” 

How  to  Evaluate  No  Past  Performance 

In  most  cases  the  evaluation  group  will  find  some  related  government  or  other  public  or  private 
past  performance  information  for  each  contractor  and  subcontractor.  Such  information  will 
usually  surface  if  the  evaluation  approach  allows  a  broad  interpretation  of  relevancy  or  takes  into 
account  information  regarding  the  past  performance  of  predecessor  companies,  key  personnel 
who  have  relevant  experience,  or  subcontractors  that  will  perform  key  aspects  of  the 
requirement. 

Occasionally,  however,  an  evaluation  group  cannot  find  any  relevant  information.  In  those 
cases,  you  must  treat  an  offeror’s  lack  of  past  performance  as  an  unknown  performance  risk, 
having  no  positive  or  negative  evaluative  significance.  This  allows  the  government  to  evaluate 
past  performance  in  a  manner  that  is  fair  to  newcomers.  It  neither  rewards  nor  penalizes  firms 
without  relevant  performance  history.  The  method  and  criteria  for  evaluating  offerors  with  no 
relevant  past  performance  information  should  be  constructed  for  each  specific  acquisition  to 
ensure  that  such  offerors  are  not  evaluated  favorably  or  unfavorably  on  past  performance. 

You  may  use  a  variety  of  rating  methods  to  evaluate  offerors  with  no  past  performance  history. 
Regardless  of  the  method  selected,  the  solicitation  must  clearly  describe  the  approach  that  will  be 
used  for  evaluating  offerors  with  no  relevant  performance  history. 
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Encourage  newcomers  to  identify  other 
related  past  performance  information 


You  can  also  ease  the  impact  on  newcomers  by  including  language  in  the  solicitation  that 
encourages  them  to  identify  related  past  performance  information  for  Federal,  state  and  local 
government  contracts,  commercial  contracts,  key  personnel,  and  major  subcontractors. 

What  to  Include  in  the  Solicitation 


Include  this  form  with  the  solicitation’s  instructions  to  offerors  to  simplify  the  submission  and 
evaluation  of  past  performance  information 

(To  be  completed  by  the  offeror ) 

1.  Contract  Number:  _ _ 

2.  Contractor  (Name,  Address  and  Zip  Code): 

3.  Type  of  Contract:  Negotiated _ Sealed  Bid _ 

Fixed  Price _  Cost  Reimbursement _  Hybrid  (explain) _ 

4.  Complexity  of  Work:  Difficult _  Routine _ 

5.  Description,  location  &  relevancy  of  work: 

6.  Contract  Dollar  Value:  _ 

Status:  Active _  Complete _ 

7.  Date  of  Award:  _ 

Contract  Completion  Date  (including  extensions):  _ 

8.  Type  and  Extent  of  Subcontracting: 
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9.  Name,  Address,  Tel.  No.  &  e-mail  of  the  Procuring  Contracting  Officer  and/or  the 
Contracting  Officer’s  Representative  (COR)  (and  other  references,  e.g.,  Administrative 
Contracting  Officer,  if  applicable): 


Blueprint:  Format  for  Submission  of  Past  Performance  Information 

The  solicitatipn,  at  a  minimum,  must  clearly  describe  the  approach  you  will  use  to  evaluate  past 
performance.  The  approach  must  include  what  past  performance  information  you  will  evaluate, 
how  it  will  be  evaluated,  its  weight  or  relative  importance  to  the  other  evaluation  factors  and 
subfactors,  and  how  you  will  evaluate  offerors  with  no  past  performance  history.  The  amount  of 
information  you  request  should  be  tailored  to  the  circumstances  of  the  acquisition.  The  amount 
should  be  reasonable  and  not  impose  excessive  burdens  on  offerors  or  evaluators.  The  proposal 
evaluation  information,  as  a  minimum,  should  clearly  state  that: 

•  The  government  will  conduct  a  performance  risk  assessment  based  upon  the  past 
performance  of  the  offerors  and  their  proposed  major  subcontractors  as  it  relates  to 
the  probability  of  successfully  performing  the  solicitation  requirements; 

•  In  conducting  the  performance  risk  assessment,  the  government  may  use  data 
provided  by  the  offeror  and  data  obtained  from  other  sources;  and 

•  While  the  government  may  elect  to  consider  data  obtained  from  other  sources,  the 
burden  of  providing  thorough  and  complete  past  performance  information  rests  with 
the  offeror. 

The  proposal  submission  instructions  must,  as  a  minimum,  instruct  offerors  to  submit  recent  and 
relevant  information  concerning  contracts  and  subcontracts  (including  Federal,  State,  and  local 
government  and  private)  that  demonstrate  their  ability  to  perform  the  proposed  effort. 

Rather  than  having  the  offeror  submit  a  lengthy  narrative  response,  the  information  requested 
should  be  limited  to  a  half-page  summary  of  the  offeror’s  performance  claim  for  each  contract  or 
subcontract.  The  summary  should  include  contract  numbers,  contract  type,  description  and 
relevancy  of  the  work,  dollar  value,  contract  award  and  completion  dates,  and  names,  phone 
numbers,  and  e-mail  addresses  for  references  in  contracting  and  technical  areas. 

In  addition,  offerors  should  be  given  the  opportunity  to  explain  why  they  consider  the  contracts 
they  have  referenced  as  being  relevant  to  the  proposed  acquisition.  Relevancy  is  defined  as 
something  that  has  a  logical  connection  with  the  matter  under  consideration.  (See  section 
entitled,  Data  Relevancy.)  The  instructions  should  also  permit  offerors  to  provide  information 
on  problems  encountered  on  such  contracts  and  the  actions  taken  to  correct  such  problems.  Also, 
it  is  important  that  the  offeror  specifically  describe  the  work  that  its  major  subcontractors  will 
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long. 

•  If  a  problem  surfaced,  ask  what  the  government  and  contractor  did  to  fix  it. 

•  Ask  for  a  description  of  the  types  of  personnel  (skill  and  expertise)  the  contractor  used  and  the 
overall  quality  of  the  contractor’s  team.  Did  the  company  appear  to  use  personnel  with  the 
appropriate  skills  and  expertise? 

•  Ask  how  the  contractor  performed  considering  technical  performance  or  quality  of  the  product 
or  service;  schedule;  cost  control  (if  applicable);  business  relations;  and  management. 

•  Ask  whether  the  contractor  was  cooperative  in  resolving  issues. 

•  Inquire  whether  there  were  any  particularly  significant  risks  involved  in  performance  of  the 
effort. 

•  Ask  if  the  company  appeared  to  apply  sufficient  resources  (personnel  and  facilities)  to  the 
effort. 

•  If  the  company  used  subcontractors,  ask:  What  was  the  relationship  between  the  prime  and 
subcontractors?  What  was  the  management  role  of  the  prime  and  how  well  did  it  manage  the 
subcontractors?  Did  the  subcontractors  perform  the  bulk  of  the  effort  or  just  add  breadth  or 
depth  on  particular  technical  areas?  If  the  subcontractors  worked  on  specific  technical  areas, 
what  were  those  areas  and  why  were  they  accomplished  by  the  subcontractors  rather  than  the 
prime? 

•  If  a  problem  is  uncovered  that  the  reference  is  unfamiliar  with,  ask  for  another  individual  who 
might  have  the  information. 

•  Ask  if  this  firm  has  performed  other  past  efforts  with  the  reference’s  agency. 

•  Ask  about  the  company’s  weak  points  or  what  the  reference  liked  least. 

•  Inquire  whether  the  reference  has  any  reservations  about  recommending  a  future  contract 
award  to  this  company. 

•  Inquire  whether  the  reference  knows  of  anyone  else  who  might  have  past  performance 
information  on  the  offeror. 


Blueprint:  Typical  Past  Performance  Questions 

Where  to  Conduct  Telephone  Interviews 

Following  the  screening  of  previous  contracts  for  further  in-depth  review,  each  evaluation  group 
member  should  send  questionnaires  and/or  initiate  telephone  calls  to  the  identified  references  for 
those  efforts.  The  interviewing  and  reporting  of  results  are  usually  individual  efforts  conducted 
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by  each  evaluation  group  member.  However,  it  is  sometimes  helpful  to  collect  information  as  a 
group  through  the  use  of  conference  calls.  In  any  event,  the  environment  in  which  this  work  is 
done  significantly  impacts  both  the  time  required  to  complete  this  portion  of  the  process  and  the 
quality  of  the  results.  These  activities  are  hampered  severely  if  each  group  member  attempts  to 
conduct  telephone  interviews  at  their  normal  work  site  with  all  of  its  attendant  interruptions, 
distractions,  and  security  risks. 

If,  on  the  other  hand,  the  group  members  are  able  to  assemble  as  a  group  for  telephone 
interviews,  they  will  be  able  to  provide  considerable  reinforcement  and  instant  feedback  for  one 
another.  Group  members  should  be  able  to  devote  their  undivided  attention  to  this  initial 
assessment  process.  Although  this  approach  requires  a  secure  area  that  is  large  enough  to 
accommodate  all  of  the  group  members,  the  resulting  benefits  are  significant. 

How  to  Conduct  Telephone  Interviews 

The  telephone  interview  process  is  an  art  form.  Until  a  smooth  conversation  pattern  is 
developed,  it  is  an  inherently  uncomfortable  situation  for  many  people.  There  will  be  some 
difficulty  learning  how  to  start  a  telephone  interview,  keep  it  moving,  and  cover  all  important 
areas.  As  the  interviewing  process  continues,  the  evaluation  group  member  usually  uncovers 
special  items  of  interest  that  he  or  she  will  want  to  pursue  through  follow-up  calls. 

At  least  two  references  should  be  contacted  on  each  previous  contract  effort  selected  for  in-depth 
review.  The  contracting  officer,  program  manager,  and  contracting  officer’s  representative  often 
prove  to  be  excellent  sources  of  information.  Additional  references  are  often  identified  during 
the  interviews.  Maximum  effectiveness  occurs  when  the  expertise  of  the  evaluation  group 
interviewer  matches  that  of  the  reference. 

Prior  to  initiating  a  telephone  interview,  a  group  member  should  gather  all  available  information 
on  a  specific  effort  and  draft  a  list  of  questions.  There  may  be  a  common  group  of  questions  for 
all  offerors  and/or  tailored  questions  for  each  offeror,  depending  upon  the  circumstances.  These 
questions  can  either  be  sent  as  questionnaires  to  each  reference  or  be  used  by  the  group  member 
during  the  telephone  interview. 

At  the  start  of  each  telephone  interview,  the  group  member  should  explain  the  purpose  of  the  call 
and  request  voluntary  assistance  from  the  reference.  The  interviewer  should  explain  that  he  or 
she  will  document  the  results  of  the  conversation  and  send  a  copy  of  the  memorandum  to  the 
reference  for  verification.  There  is  usually  no  need  to  divulge  the  solicitation  number,  program 
description,  or  other  identifying  information  to  the  reference.  If  you  do  so,  you  need  to  obtain  a 
nondisclosure  statement. 

In  most  instances  the  reference  will  willingly  provide  the  information  requested.  In  those  rare 
cases  when  the  reference  is  reluctant  to  participate,  the  interviewer  should  assure  the  reference  of 
anonymity.  At  the  least,  the  reference  should  be  requested  to  provide  additional  references. 

It  is  important  to  pursue  and  document  the  underlying  facts  supporting  any  conclusionary 
statements  received  on  a  contractor.  The  evaluation  group  member  can  determine  neither  the 
magnitude  of  a  reported  problem  nor  its  possible  impact  on  the  current  risk  assessment  without 
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receive  the  greatest  consideration.  The  assessment  should  include  a  description  of  the  underlying 
rationale  for  the  conclusions  reached.  As  long  as  that  rationale  is  reasonable,  it  will  withstand 
scrutiny  even  if  other  reasonable  conclusions  exist. 

A  word  of  caution  is  appropriate  concerning  offeror  promises  to  correct  past  performance 
failures,  as  opposed  to  actions  already  taken  to  correct  such  failures.  A  promise  to  improve  does 
not,  by  itself,  improve  past  performance.  However,  demonstrated  corrective  actions  reflect  a 
commitment  to  rectify  past  performance  problems,  and  therefore,  can  lower  the  risk  of  similar 
performance  failures. 

Disclosing  Past  Performance  Information  to  Offerors 

The  contracting  officer  must  also  provide  offerors  with  the  opportunity  to  comment  on  negative 
past  performance  information  on  which  offerors  have  not  had  a  previous  opportunity  to 
comment.  This  practice  ensures  fairness  for  the  competing  offerors.  The  validation  process  is 
particularly  important  when  the  negative  information  is  provided  by  only  one  reference,  or  when 
there  is  any  doubt  concerning  the  accuracy  of  the  information.  Usually,  negative  information 
reflects  performance  that  was  less  than  satisfactory,  although  this  is  a  judgment  call  that  will 
depend  upon  the  circumstances  of  the  acquisition.  Note  that  while  the  government  must  disclose 
past  performance  problems  to  offerors  it  shall  not  disclose  the  names  of  individuals  who 
provided  information  about  an  offeror’s  past  performance. 

A  special  problem  arises  with  respect  to  subcontractors.  Past  performance  information 
pertaining  to  a  subcontractor  cannot  be  disclosed  to  a  private  party  without  the  subcontractor’s 
consent.  Because  a  prime  contractor  is  a  private  party,  the  government  needs  to  obtain  the 
subcontractor’s  consent  before  disclosing  its  past  performance  information  to  the  prime  during 
negotiations.  There  are  a  variety  of  ways  to  obtain  subcontractor  consent.  For  example,  the 
solicitation  could  require  the  prime  to  submit  its  subcontractor’s  consent  along  with  the  prime’s 
proposal  to  the  government. 

What  to  Include  in  the  Assessment  Report 

The  goal  is  to  avoid  saying  too  much  or  too  little  in  the  evaluation  report.  Although  there  is  no 
need  to  restate  everything  contained  in  the  telephone  memoranda,  the  evaluation  group  must 
provide  the  source  selection  authority  with  sufficient  information  to  make  informed  judgments. 

Conclusionary  statements  must  be  supported  by  the  underlying  factual  basis.  The  best  practice  is 
to  state  the  conclusion  and  provide  specific  examples  that  support  that  conclusion. 

To  ensure  that  the  risk  assessments  provide  the  necessary  background  information  and  are 
structured  consistently,  the  entire  evaluation  group  should  review  and  evaluate  the  report  on  each 
offeror.  During  this  review,  the  evaluation  group  should  correct  statements  that  appear 
unsupported,  inconsistent,  or  unnecessary. 

Occasionally  the  evaluation  group  will  be  unable  to  arrive  at  a  unanimous  agreement  on  a 
particular  risk  assessment.  If  this  occurs,  the  evaluation  group  may  include  the  dissenting 
opinion  as  part  of  the  assessment  report. 
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Reporting  the  Past  Performance  Evaluation  Results 

The  evaluation  group’s  submission  of  the  assessment  report  usually  completes  the  major  portion 
of  its  work.  The  evaluation  group  leader,  should  remind  the  source  selection  official  of  the 
purpose  of  the  group  and  the  past  performance  evaluation  approach,  including  offerors  with  no 
past  performance  history,  described  in  the  solicitation.  This  is  to  ensure  that  everyone  fully 
comprehends  the  significance  of  the  results  being  reported.  Experience  reveals  that  source 
selection  officials  are  more  apt  to  rely  upon  evaluation  group  results  if  they  thoroughly 
understand  the  process. 


How  to  Treat  Past  Performance  Information 

Information  concerning  the  past  performance  of  an  offeror  or  of  its  proposed  subcontractors 
should  be  treated  as  sensitive  source  selection  information.  This  information  sometimes  includes 
information  that  is  proprietary,  such  as  trade  secrets  and  confidential  commercial  or  financial 
data  that  would  not  be  released  under  the  Freedom  of  Information  Act.  Current  laws, 
regulations,  and  policies  governing  storage,  access,  disclosure,  and  marking  of  source  selection 
and  proprietary  information  must  be  observed  at  all  times.  Questions  concerning  the  procedures 
for  the  handling  of  past  performance  information  should  be  referred  to  the  contracting  officer  or 
legal  counsel  for  resolution. 

The  evaluation  group  must  retain  the  records  of  its  activity  throughout  the  source  selection 
process.  Upon  contract  award  or  cancellation  of  the  solicitation,  all  evaluation  group  records  are 
provided  to  the  contracting  officer  for  retention  along  with  the  other  source  selection  documents. 

Improving  the  Evaluation  Process 

The  methods  described  in  this  guide  will  evolve  as  our  needs  change  and  as  our  knowledge  base 
expands.  It  is  important  for  all  of  us  to  share  lessons  learned  in  this  process.  You  can  help 
others  benefit  from  your  experiences  by  submitting  lessons  learned  via  the  Internet  at 
http://www.army-acquisition.net/amcweb/ssp/ssp_ap_d.htm#57 
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Guidelines  for  Conducting  Debriefings 


Overview 


This  appendix  provides  easy  to  follow  guidance  for  conducting  both  preaward  and  postaward 
debriefings.  It  should  always  be  used  in  conjunction  with  the  basic  best  value  guide.  Using  this 
guidance  will  result  in  meaningful  debriefings  and  will  instill  greater  confidence  in  the 
acquisition  process.  Comprehensive  and  open  debriefings  will  strengthen  and  enhance  our 
relationship  with  industry. 


What  Is  A  Debriefing? 

A  debriefing  is  a  meeting  between  government  personnel  and  an  offeror  who  has  been  eliminated 
from  the  competition  either  prior  to  or  after  contract  award.  The  purposes  of  a  debriefing  are: 

•  To  explain  the  rationale  for  exclusion  from  the  competition. 

•  To  instill  confidence  in  the  offeror  that  it  was  treated  fairly. 

•  To  assure  the  offeror  that  proposals  were  evaluated  in  accordance  with  the 
solicitation  and  applicable  laws  and  regulations. 

•  To  identify  weaknesses  in  the  offeror’s  proposal  so  the  offeror  can  prepare  better 
proposals  in  future  government  acquisitions. 

•  To  reduce  misunderstandings  and  protests. 

•  To  give  the  offeror  an  opportunity  to  provide  feedback  regarding  the  solicitation, 
discussions,  evaluation,  and  the  source  selection  process. 

A  debriefing  is  NOT: 

•  A  page-by-page  analysis  of  the  offeror’s  proposal. 

•  A  comprehensive  point-by-point  comparison  of  the  proposals  of  the  debriefed 
offeror  and  the  successful  offeror(s). 

•  A  debate  or  defense  of  the  government’s  award  decision  or  evaluation  results. 


Debriefings  enhance  understanding 
of  the  selection  process 


Who  Is  Entitled  To  A  Debriefing? 
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Whenever  the  Government  awards  a  contract  based  on  competitive  proposals,  an  unsuccessful 
offeror  is  entitled  to  a  debriefing  if  it  timely  requests  one.  Debriefings  are  not  required  when 
sealed  bidding  procedures  are  used. 

Offerors  excluded  from  the  competitive  range  or  otherwise  excluded  from  the  competition  before 
award  may  request  a  preaward  or  postaward  debriefing. 

Good  business  practice  dictates  that  the  Government  should  debrief  the  awardee  if  requested. 
Although  the  Government  often  conducts  a  postaward  conference  with  the  awardee,  that  meeting 
does  not  necessarily  substitute  for  a  debriefing. 

When  Must  A  Debriefing  Be  Conducted? 

The  Government  must  conduct  a  debriefing  for  an  unsuccessful  offeror  if: 

(1)  the  offeror  makes  a  written  request  for  a  debriefing,  and 

(2)  the  request  is  received  by  the  contracting  activity  within  3  days  after  the  offeror 
received  notice  of  exclusion  from  the  competition  or  contract  award. 

Here's  a  simple  rule  for  computing  time  periods:  Count  days  as 
calendar  days  and  include  weekends  and  legal  holidays.  Don 't 
count  the  day  the  offeror  received  the  notice.  Start  with  the  next  day. 

For  example,  if  the  offeror  received  the  notice  on  Tuesday,  the  1st 
day  is  Wednesday,  the  2d  day  is  Thursday,  and  the  3d  day  is  Friday. 

Thus,  you  must  receive  the  request  for  the  debriefing  no  later  than 
close  of  business  on  Friday. 

Establishing  the  date  the  offeror  received  the  notice  may  be  difficult  if  the  notice  is  sent  by 
regular  mail.  Accordingly,  you  should  consider  sending  the  notice  by  mail  with  return  receipt 
requested  or  by  electronic  transmission  (fax)  with  immediate  acknowledgment  requested. 
Remember,  every  day  of  delay  in  notifying  the  offeror  usually  extends  by  one  day  the  time  in 
which  a  protest  may  be  filed. 

If  the  request  is  for  a  preaward  debriefing,  the  contracting  officer  shall  make  every  effort  to 
debrief  the  unsuccessful  offeror  as  soon  as  practicable.  If  there  are  compelling  reasons  that  it  is 
not  in  the  Government’s  best  interest  to  conduct  the  debriefing  prior  to  award,  the  contracting 
officer  can  delay  it  no  later  than  the  time  when  postaward  debriefings  are  provided.  In  such 
cases,  you  must  document  the  contract  file  with  the  rationale  for  the  delay.  This  debriefing 
should  then  be  conducted  as  a  postaward  debriefing. 

If  an  offeror,  who  has  been  eliminated  from  the  competition  prior  to  award  requests  a  delayed 
debriefing  (i.e.,  postaward),  the  date  the  offeror  knew  or  should  have  known  the  basis  of  a  protest 
shall  be  the  date  the  offeror  received  notice  of  the  exclusion  from  the  competition. 

If  you  receive  a  timely  request  for  a  postaward  debriefing,  you  must  conduct  the  debriefing 
within  5  days,  to  the  maximum  extent  practicable,  after  receipt  of  the  request.  Remember  that 
the  scheduling  of  a  debriefing  may  have  significant  ramifications  if  a  protest  is  later  filed.  A 
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protester  filing  a  protest  with  the  General  Accounting  Office  is  entitled  to  an  automatic 
suspension  of  contract  performance  if  the  protest  is  filed  within  5  days  of  the  debriefing  date 
offered  to  the  protester  (or  10  days  after  contract  award,  whichever  is  later). 

Thus,  it  is  extremely  important  that  you  schedule  the  debriefing  for  the  earliest  possible  date  for 
the  Government.  You  should  inform  the  offeror  of  the  scheduled  date  in  writing  by  electronic 
means  with  immediate  acknowledgment  requested.  If  the  offeror  is  unable  to  attend  the 
scheduled  date  and  requests  a  later  date,  the  offeror  should  be  required  to  acknowledge  in  writing 
that  it  was  offered  an  earlier  date,  but  requested  the  later  date  instead.  This  procedure  will 
protect  the  Government’s  interests  if  the  offeror  subsequently  files  a  protest. 

If  an  offeror  submits  an  untimely  request  for  debriefing  (i.e.,  received  more  than  3  days  after 
notice  of  elimination  from  the  competition  or  contract  award),  the  contracting  officer  should 
nonetheless  conduct  a  debriefing  if  feasible.  However,  the  contracting  officer  should  inform  the 
offeror  that  the  request  is  untimely.  Untimely  requests  do  not  invoke  the  statutory  provision  for 
automatic  suspension  of  contract  performance  unless  the  protest  is  filed  within  10  days  of 
contract  award. 

The  time  periods  stated  in  this  guidance  were  current  on  the  date  of  publication.  However, 
always  check  with  your  legal  counsel  for  the  latest  General  Accounting  Office  rules  and 
revisions. 


Where  Is  A  Debriefing  Held? 

The  contracting  officer  is  responsible  for  selecting  a  suitable  location  for  the  debriefing  that 
ensures  a  professional  presentation  in  a  nondistracting  environment. 

A  conference  room  will  normally  suffice:  Obviously,  the  room  must  be  equipped  with  the 
necessary  audiovisual  equipment  that  will  be  used  during  the  debriefing.  The  contracting  officer 
should  consult  with  the  offeror  to  ensure  that  adequate  seating  will  be  available  for  all  personnel 
attending.  If  classified  material  will  be  discussed,  appropriate  arrangements  must  be  made. 

Although  face-to-face  debriefings  are  frequently  preferable,  it  is  appropriate  to  conduct 
debriefings  by  telephone  or  other  electronic  means.  Sometimes,  it  will  be  financially  prohibitive 
for  the  offeror  to  attend  in  person.  The  needs  of  the  offeror  should  be  afforded  due 
consideration,  but  the  contracting  officer  makes  the  final  decision  as  to  the  debriefing  location. 

If  some  of  the  government  personnel  are  located  at  an  installation  other  than  the  contracting 
office,  the  contracting  officer  may  decide  that  those  individuals’  attendance  is  best  accomplished 
by  telephone  or  video  conference.  The  possibilities  are  endless  --  make  use  of  available 
technology  and  do  what  makes  sense! 

Who  Attends  A  Debriefing? 


Government  Personnel 


The  contracting  officer  is  responsible  for  chairing  the  debriefing.  This  doesn’t  mean  that  the 
contracting  officer  must  conduct  the  entire  debriefing.  In  fact,  usually  the  contracting  officer 
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will  rely  on  government  technical  and  cost/price  personnel  to  present  the  portions  of  the 
debriefing  that  address  those  specialized  areas  of  the  offeror’s  proposal.  The  debriefing  should 
be  conducted  with  only  one  offeror  at  a  time. 

The  contracting  officer  identifies  the  Government  debriefing  team  members.  The  selection  is 
based  on  the  complexities  presented  in  each  acquisition.  The  key  is  to  ensure  that 
knowledgeable  Government  personnel  are  present.  Because  of  the  statutory  requirement  for  a 
prompt  debriefing,  the  contracting  officer  should  tentatively  select  the  team  before  the  contract 
award  is  announced. 

When  determining  the  composition  of  the  team,  you  must  keep  in  mind  the  objectives  of  a 
meaningful  debriefing.  Above  all,  the  Government  must  display  that  it  fully  understood  the 
offeror’s  proposal.  If  this  is  not  conveyed,  the  offeror  will  obviously  have  little  confidence  in  the 
conduct  of  the  acquisition.  Many  debriefings  have  failed  merely  because  the  contracting  officer 
did  not  ensure  appropriate  Government  personnel  were  present. 


Involve  government  legal  counsel  in  debriefings 


The  contracting  officer’s  legal  counsel  should  attend  the  debriefing.  Legal  counsel  should  also 
assist  in  the  preparation  of  the  debriefing.  If  the  acquisition  is  in  litigation  or  under  protest,  legal 
counsel  must  attend  the  debriefing.  Legal  counsel  should  be  informed  of  any  indicators  that  a 
protest  is  likely.  However,  the  contracting  officer  will  not  deny  a  debriefing  because  a  protest  is 
threatened  or  has  already  been  filed. 

Debriefed  Offeror  Personnel 

The  contracting  officer  should  ask  an  offeror  scheduled  for  a  debriefing  to  identify  all  individuals 
by  name  and  position  who  will  attend  the  debriefing.  Normally,  no  limitation  should  be  placed 
on  the  personnel  the  debriefed  offeror  may  bring  to  a  debriefing.  Nonetheless,  space  limitations 
of  Government  facilities  may  require  restrictions  in  extraordinary  cases  on  the  number  of  offeror 
personnel  invited  to  attend.  The  contracting  officer  should  not  impose  restrictions  unless  the 
contracting  officer  has  determined  that  all  suitable  alternate  facilities  are  unavailable. 

There  may  be  times  when  you  should  consider  offering  to  have  high  level  officials  from  the 
requiring  activity  present  at  the  debriefing  if  the  offeror  in  turn  agrees  to  have  commensurate 
management  officials  present.  This  has  proven  effective  when  high  level  officials  were 
substantially  involved  in  the  acquisition  or  the  acquisition  received  extensive  high-level  visibility 
or  scrutiny. 


Preparing  For  A  Debriefing 


A  debriefing  requires  preparation! 
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Only  the  foolhardy  will  attempt  to  conduct  a  debriefing  unprepared.  Experience  has  shown  that 
going  into  a  debriefing  unprepared  is  the  surest  way  to  lose  the  confidence  of  the  offeror  and 
increase  the  prospects  of  a  protest. 

Because  debriefings  are  time  sensitive,  preparation  must  begin  before  proposal  evaluation  is 
complete.  Usually,  the  proposal  evaluation  board  will  assist  you  in  preparing  debriefing  charts 
and  conducting  the  debriefing.  Accordingly,  at  the  time  the  evaluation  board  is  formed,  you 
must  inform  the  evaluators  that  their  duties  include  assisting  with  debriefings. 

Preparation  for  the  debriefing  should  be  thorough.  Debriefings  permit  offerors  to  learn  their 
strengths  and  weaknesses  and  how  to  improve  future  Government  proposals.  Offerors  may  also 
rely,  however,  on  these  sessions  to  influence  their  decision  regarding  filing  a  protest.  An 
effective  debriefing  can  often  deter  a  protest  by  demonstrating  that  the  Government  conducted  a 
thorough,  fair  evaluation  and  made  a  sound  decision  according  to  the  established  solicitation 
evaluation  methodology. 

The  extent  of  preparation  necessary  varies  considerably  with  the  complexity  of  each  acquisition. 
Sometimes,  merely  preparing  debriefing  charts  is  sufficient.  Other  times,  dry  run  rehearsals  are 
necessary. 

Finally,  all  government  personnel  attending  the  debriefing  must  be  briefed  on  their  roles  and 
expected  demeanor  during  the  debriefing.  Argumentative  or  overly  defensive  conduct  should  be 
discouraged,  and  Government  personnel  should  be  instructed  to  make  a  positive  presentation. 

What  Materials  Should  Be  Brought  Into  the  Debriefing  Room? 

Debriefing  material  normally  consists  of  briefing  charts  and  notes  prepared  for  use  during  the 
debriefing. 

•  Experience  has  shown  that  Government  personnel  should  Not  bring  the  following 
materials  into  the  debriefing  room: 

•  Proposals  of  other  offerors,  and 

•  Proposal  evaluation  reports  of  other  offerors’  proposals. 

The  presence  of  these  documents  can  lead  to  the  inadvertent  disclosure  of  proprietary 
information  and  prohibited  point-by-point  comparisons  of  proposals. 

Experience  has  also  shown  that  a  faulty  memory  or  misstatements  by  Government  personnel  are 
detrimental  to  a  successful  debriefing.  Thus,  contracting  officers  must  ensure  that  necessary 
notes  or  other  documents  are  accessible  during  the  debriefing. 

Debriefing  the  Unsuccessful  Offeror 
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SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 


High  Performance  Computer  Contract 

DAAX08-XX-C-1234 


Unsuccessful  Coip.,Inc. 

Debriefing 

June  2,  XXXX 


Typical  Debriefing  Chart  No.  1  -  Cover  Page 


SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 

AGENDA 

•  Introduction 

•  Ground  Rules 

•  Source  Selection  Process 

•  Evaluation  Criteria 

•  Rationale  for  Award  Decision 

•  Overall  Ranking  of  Proposals 

•  Relevant  Questions 

Typical  Debriefing  Chart  No.  2  -  Agenda 
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SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 


SOURCE  SELECTION  PROCESS 
Source  Selection  Team 


Source  Selection  Authority 

Proposal  Eva 

i 

iluation  Board 

! 

i 

Technical 

Cost 

Past  Performance 

Typical  Debriefing  Chart  No.  3  -  Source  Selection  Process 
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SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 


EVALUATION  RATINGS  DEFINITIONS 


EXCELLENT:  The  proposal  demonstrates  an  excellent  understanding  of  requirements  and  approach 
that  significantly  ex  ceeds  performance  or  capability  standards.  It  has  ex  ceptional  strengths  that  will 
significantly  benefit  the  Government. 

GO  OD :  The  pr  opo  sal  dem  onsir  ate  s  a  go  od  understanding  of  the  re  quirem  ent s  and  an  appr  oach  that 
ex  ceeds  performance  or  capability  standards.  It  has  one  or  more  strengths  that  w*ill  benefit  the 
Government. 

SATISFACTORY:  The  proposal  demonstrates  an  acceptable  understanding  of  requirements  and  an 
approach  that  meets  performance  or  capability  standards.  It  demonstrates  an  acceptable  sdutior; 
however,  has  few  or  no  strengths. 

MARGINAL:  The  proposal  demonstrates  shallow  understanding  of  the  requirements  and  an  approach 
that  only  margnally  meets  performance  or  capability  standards  necessary  for  minimal  but  acceptable 
contract  performance. 

UNSATISFACTORY:  The  proposal  fails  to  meet  performance  and  capability  standards.  The 
requirements  can  only  be  met  withmaj  or  changes  to  the  proposal. 

Typical  Debriefing  Chart  No.  4  -  Evaluation  Ratings 
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SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 


FACTOR  EVALUATION  RESULTS 


Unsuccessful  Successful 

Corp.  Corp. 


Technical  Merit 

Good 

Excellent 

significantly  more  important  than 

Proposal  Risk 

Moderate 

Low 

which  is  equal  to 

Past  Performance  Risk 

Low  Risk 

Low  Risk 

which  is  equal  to 

Cost 

$1,095,200 

$1,186,100 

Typical  Debriefing  Chart  No.  5  -  Evaluation  Results 
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SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 

STRENGTHS/WEAKNESSES 
Unsuccessful  Corp. 

SIGNIFICANT  STRENGTHS 

♦  Outstanding  hardware  architecture. 

•  Excellent  cross  utilization  of  personnel. 

♦  Extensive  corporate  experience  in  high  performance  computer  hardware. 

•  Outstanding  internal  control  and  quality  assurance  procedures. 

•  Use  of  verification  requirements  traceability  matrix. 

♦  Excellent  understanding  of  data  requirements. 

•  Excellent  understanding  of  unique  platform  requirements. 

SIGNIFICANT  WEAKNESSES 

♦  Software  is  not  user  friendly. 

♦  Lack  of  corporate  experience  in  software  development. 

•Insufficient  detail  on  possibility  of  concurrent  installation/training. 

•  Part-time  commitment  of  some  key  personnel  to  other  projects. 

Typical  Debriefing  Chart  No.  6  -  Strength s/W eakn e s s es 
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SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 

TECHNICAL  MERIT  SUBFACTORS 


Unsuccessful  Successful 

Corp.  Corp. 


Technical  Merit 

Good 

Excellent 

Hardware  Design 

Excellent 

Excellent 

Software  Configuration 

Satisfactory 

Excellent 

Corporate  Experience 

Excellent 

Excellent 

■pin  jrr*i T*r*r>  a  nr>  TATTiT 

ALL  ELEMENTS  ARE  EQUAL 


Typical  Debriefing  Chart  No.  7  -  Technical  Merit 
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SOURCE  SELEC TION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 


PROPOSAL  RISK  SUBFACTORS 


Unsuccessful  Successful 

Corp.  Corp. 


ALL  ELEMENTS  ARE  EQUAL 


Typical  Debriefing  Chart  No.  8  -  Proposal  Risk 
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SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts-  Post  Award 


PAST  PERFORMANCE  SUBFACTORS 


L 

Unsuccessful  Successful 

Corp.  Corp. 

Performance  Risk 

Low  Risk 

Low  Risk 

Quality  of  Performance 

Low  Risk 

Low  Risk 

Cost  Management 

Moderate  Risk 

Low  Risk 

Contract  Schedules 

Low  Risk 

Moderate  Risk 

Customer  Relations 

Low  Risk 

ALL 

Low  Risk 

ELEMENTS  ARE  EQUAL 

Typical  Debriefing  Chart  No.  9  -  Performance  Subfactors 


192 


Typical  Debriefing  Chart  No.  10  -  Cost 
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SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 

COST  REALISM  ADJUSTMENTS 
Unsuccessful  Corp. 


Proposed  Adj.  Evaluated 

G& A  Expense  $96,678  +  $8,320  $104,998 
Travel  $  4,600  +$12,680  $17,280 

Typical  Debriefing  Chart  No.  11  -  Cost  Realism  Adjustments 
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SOURCE  SELECTION “BLUEPECNT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 


RATIONALE  FOR  AWARD  DECISION 


Successful  Corp’s  innovative  approach  to  software 
configuration  represents  a  significant  advancement  in  the  state 
of  the  art  in  terms  of  capability  and  user  friendly  interface.  Its 
proposed  hardware  architecture  is  superior  to  the  other  offerors 
in  terms  of  processing  speed.  Successful  Corp.  has  extensive 
experience  in  the  development  of  high  performance  computer 
systems  and  its  past  performance  on  similar  contracts  was 
excellent  Overall,  Successful  Corp’s  proposed  solution 
coupled  with  its  low  risk  represents  the  best  value  to  the 
Government  despite  its  slightly  higher  cost 

Typical  Debriefing  Chart  No.  12  -  Aware  Decision  Rationale 


SOURCE  SELECTION  “BLUEPRINT” 

Typical  Unsuccessful  Offeror  Debriefing  Charts  -  Post  Award 


OVERALL  RANKING  OF  PROPOSALS 


1.  Successful  Corp. 

2.  Offeror  C 

3.  Unsuccessful  Corp. 

4.  Offeror  E 

5.  Offeror  B 

6.  Offeror  A 


Typical  Debriefing  Chart  No.  13  -  Proposal  Rankings 

Blueprint:  Typical  Unsuccessful  Offeror  Debriefing  Charts  —  Post  Award 

Whos  in  Charge? 

The  contracting  officer  is  responsible  for  the  debriefing  and  as  such  must  never  lose  control  of 
the  debriefing. 


Introduction 

The  debriefing  should  begin  with  a  brief  introduction  of  all  attendees.  You  should  then  explain 
the  purposes  of  the  debriefing.  (Refer  to:  What  is  a  debriefing?) 

Ground  Rules 

Next,  you  should  inform  everyone  of  the  ground  rules  for  the  debriefing,  any  time  constraints, 
and  the  debriefing  agenda. 


Handling  Questions 

During  both  preaward  and  postaward  debriefings,  the  debriefed  offeror  is  permitted  to  ask 
relevant  questions  pertaining  to  whether  the  Government  followed  the  source  selection 
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procedures  set  forth  in  the  solicitation,  applicable  regulations,  and  other  applicable  authorities. 
Often,  contracting  officers  request  that  questions  from  the  debriefed  offeror’s  personnel  be 
tunneled  to  the  debriefed  offeror’s  main  spokesperson  who  will  ask  all  the  questions.  This 
practice  is  permissible  and,  in  fact,  often  enhances  the  orderly  conduct  of  a  debriefing. 

The  contracting  officer  must  make  every  effort  to  provide  reasonable  responses  to  those 
questions.  However,  caution  must  be  exercised  not  to  inadvertently  disclose  proprietary 
information  of  other  offerors. 

Normally,  responses  should  stay  within  the  confines  of  the  guidance  stated  above.  If  the 
contracting  officer  is  unprepared  to  answer  a  specific  question  at  the  debriefing,  the  contracting 
officer  should  obtain  the  answer  immediately  following  the  debriefing  and  promptly  furnish  it  to 
the  offeror.  To  avoid  this,  you  may  find  it  helpful  to  request  written  questions  ahead  of  time  to 
use  in  preparing  for  the  debriefing. 

Because  the  debriefing  is  the  forum  for  answering  reasonable  questions,  offerors  should  not  be 
invited  to  submit  questions  after  the  debriefing. 

Source  Selection  Process 

The  process  used  in  evaluating  proposals,  establishing  the  competitive  range,  and  selecting  the 
awardee  should  be  briefly  explained.  When  conducting  a  postaward  debriefing,  the  source 
selection  authority  normally  should  be  identified  by  name.  The  identities  of  the  evaluators,  other 
than  those  present  at  the  debriefing,  are  normally  not  disclosed.  Nonetheless,  the  Government 
must  convey  to  the  offeror  that  the  proposals  were  evaluated  by  qualified  personnel. 

Evaluation  Factors/Subfactors 

The  evaluation  factors  and  subfactors  disclosed  in  the  solicitation  should  be  restated  as  the 
foundation  for  discussing  the  evaluation  results. 

Evaluation  Results 

The  source  selection  information  disclosed  at  debriefings  for  an  unsuccessful  offeror  is  governed 
by  law,  regulation  and  the  sound  discretion  of  the  contracting  officer.  If  the  contracting  officer 
observes  the  following  guidelines  for  pre  and  postaward  debriefings,  there  should  be  little 
concern  about  inadvertently  disclosing  unauthorized  proprietary  information. 

Preaward  Debriefings 

At  a  minimum,  the  following  information  will  be  disclosed: 


(1)  The  evaluation  results  of  significant 
elements  in  the  debriefed  offeror’s  proposal. 
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What  are  significant  elements?  If  the  element  was  significant  enough  to  eliminate  the  offeror 
from  the  competitive  range,  it  is  probably  significant  for  debriefing  purposes.  Include  in  your 
discussion  positive  elements  of  the  offeror’s  proposal  to  help  him  improve  future  proposals. 


(2)  Summary  of  the  rationale  for  eliminating 
the  offeror  from  the  competition. 


What  Cannot  Be  Disclosed!  (Preaward) 

In  addition  to  the  information  that  may  Not  be  disclosed  in  postaward  debriefings 
(see  What  Cannot  be  Disclosed!  (Postaward)),  by  law  the  following  information 
may  not  be  disclosed  in  a  preaward  debriefing: 

(1)  The  number  of  offerors; 

(2)  The  identity  of  other  offerors; 

(3)  The  content  of  other  offeror's  proposals; 

(4)  The  ranking  of  other  offerors; 

(5)  The  evaluation  of  other  offerors. 

Postaward  Debriefings 

At  a  minimum,  the  following  information  will  be  disclosed: 


(1)  The  deficiencies  and  significant  weaknesses 
of  the  debriefed  offeror’s  proposal. 


When  is  a  weakness  significant?  If  the  weakness  was  of  significant  enough  concern  to  warrant 
its  discussion  during  the  negotiation  phase  of  the  acquisition,  it  is  probably  significant  for 
debriefing  purposes  as  well.  Whereas,  if  it  wasn’t  significant  enough  to  warrant  discussion,  it  is 
not  significant  for  debriefing  purposes  either,  unless,  of  course  the  weakness  was  created  in  the 
final  proposal  revision.  It  is  also  a  good  practice  to  discuss  the  significant  advantages  of  the 
debriefing  offeror’s  proposal. 


(2)  The  evaluation  ratings  of  the 
debriefed  offeror  and  awardee  —  but  only 
to  the  second  level  of  evaluation. 
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If  the  evaluation  board  used  adjectival  ratings,  the  adjectives  and  their  definitions  contained  in 
the  evaluation  plan  should  be  disclosed.  If  numerical  ratings  or  color  codings  were  used  instead, 
they  should  be  disclosed  likewise. 

What  is  the  second  level  of  evaluation?  For  example,  assume  the  solicitation  sets  forth  the 
following  four  evaluation  factors:  Technical,  Management,  Integrated  Logistics 

Support  (ILS),  and  Past  Performance.  This  is  the  first  level  of  evaluation  and  the  overall  ratings 
for  each  of  the  four  factors  would  be  disclosed. 

If  several  subfactors  were  separately  rated  under  a  factor  (such  as  Management  Approach, 
Proposed  Staffing,  and  Past  Corporate  Experience  in  the  Management  factor),  then  those 
subfactors  constitute  the  second  level  of  evaluation  and  their  ratings  should  also  be  released. 

The  evaluation  ratings  at  the  third  and  fourth  levels  of  evaluation  of  the  awardee’s  proposal 
should  normally  not  be  revealed.  If  ratings  are  revealed  at  these  lower  levels,  it  is  conceivable 
that  the  contracting  officer  may  run  afoul  of  the  statutory  prohibition  against  point-by-point 
comparisons  of  proposals. 

Ratings  of  the  debriefed  offeror’s  proposal  may  be  revealed  to  these  lower  levels  if  necessary  to 
explain  the  rationale  for  the  award  decision.  Be  prepared  to  explain  the  rationale  for  the  ratings 
of  the  debriefed  offeror’s  proposal. 


(3)  The  government’s  total 
evaluated  costs/prices  of  the 
debriefed  offeror’s  proposal. 


The  total  evaluated  costs/prices  of  the  debriefed  offeror’s  proposal  should  be  disclosed  for  each 
contract  line  item  (CLIN),  and  an  explanation  given  for  any  significant  cost  realism  adjustments 
made  by  the  Government  at  the  major  cost  element  level. 


(4)  The  total  evaluated  costs/prices 
of  the  awardee’s  proposal. 


The  awardee’s  total  proposed  and  evaluated  costs/prices  for  each  CLIN  should  be  disclosed. 

Do  Not  disclose  the  specific  Government  cost/price  adjustments  to  the  awardee’s  proposed 
cost/prices. 


(5)  Overall  ranking  of  all  proposals. 
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If  the  source  selection  authority  ranked  the  proposals  (i.e.,  the  best  overall  proposal,  2d  best, 
etc.),  the  overall  ranking  of  all  proposals  must  be  revealed.  However,  the  identities  of  the  other 
unsuccessful  offerors  should  not  be  revealed.  Refer  to  those  offerors  by  alphanumerical  letters 
or  other  designators. 


(6)  Rationale  for  award  decision. 


The  Government  must  disclose  a  summary  of  the  rationale  for  the  contract  award  decision.  The 
rationale  is  contained  in  the  source  selection  authority’s  (SSA)  decision  memorandum. 
Contracting  officers  should  furnish  the  debriefed  offeror  a  copy  of  the  SSA’s  memorandum,  if 
requested.  However,  evaluation  information  concerning  the  other  unsuccessful  offerors  and 
proprietary  information  of  the  award  must  be  redacted  prior  to  release. 

Identify  the  significant  advantages  of  the  awardee’s  proposal  in  general  terms  without  revealing 
confidential  proprietary  information  contained  in  the  awardee’s  proposal.  (See  below,  What 
Cannot  Be  Disclosed!) 


(7)  The  make  and  model  of  any  commercial 
end  items  proposed  by  the  awardee. 


If  the  awardee’s  proposal  includes  a  commercial  item  that  is  an  end  item  under  the  contract,  the 
make  and  model  of  the  item  must  be  disclosed. 


Information  Pertaining  To  Other 
Unsuccessful  Offerors  That  May 
Be  Released  (Postaward) 


Under  certain  circumstances,  you  may  release  other  information  such  as  the  final 
overall  ratings  for  noncost  factors  and/or  the  final  evaluated  cost/price  of  the  other 
unsuccessful  offerors.  Release  of  the  overall  noncost  ratings  is  discretionary.  However, 
release  of  the  total  final  evaluated  cost/price  is  limited  to  those  situations  where  an 
unsuccessful  offeror  consents  or  the  agency  determines  that  the  unsuccessful  offeror, 
after  consulting  with  it,  would  not  suffer  competitive  harm  from  such  a  release.  The 
decision  to  release  any  of  this  information  must  be  made  on  a  case-by-case  basis  with 
guidance  from  your  legal  office. 

What  Cannot  Be  Disclosed!  (Postaward) 

By  law  a  debriefing  may  Not  include  point-by-point  comparisons  of  the  debriefed  offeror’s 
proposal  with  the  other  proposals.  If  the  ratings  of  the  proposals  of  the  debriefed  offeror  and  the 
awardee  are  disclosed  only  to  the  second  level  of  evaluation,  this  prohibition  should  not  present  a 
problem. 
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Also  by  law,  debriefings  may  NOT  disclose  information  that  is  exempt  from  release  under  the 
Freedom  of  Information  Act  (FOIA)  relating  to: 

(1)  Trade  secrets; 

(2)  Privileged  or  confidential  manufacturing  processes  and  techniques; 

(3)  Commercial  and  financial  information  that  is  privileged  or  confidential,  including 
cost  breakdowns,  profits,  indirect  cost/rates,  and  similar  information;  and 

(4)  Names  of  individuals  providing  reference  information  about  an  offeror’s  past 
performance. 

This  information  is  normally  referred  to  as  “proprietary  information.  ” 

Proprietary  information  means  information  contained  in  a  proposal  or 
otherwise  submitted  to  the  Government  that  the  submitter  has  marked 
as  proprietary.  Proprietary  information  does  Not  include  information 
that  is  otherwise  available  without  restriction  to  the  Government  or  the 
public.  If  you  believe  that  information  marked  proprietary  is  not  truly 
proprietary,  you  should  contact  the  assigned  legal  advisor  for  an 
appropriate  determination  before  the  information  is  released. 

Debriefing  the  Awardee 

Although  debriefing  an  awardee  is  similar  in  many  respects  to  debriefing  an  unsuccessful 
offeror,  there  is  one  significant  difference  —  very  little  information  is  revealed  regarding  the 
proposals  of  the  unsuccessful  offerors. 

As  discussed  in  Debriefing  The  Unsuccessful  Offeror,  the  following  outline  is  suggested: 

•  Introduction. 

•  Explain  the  purposes  of  the  debriefing. 

•  Announce  the  ground  rules. 

•  Summarize  the  source  selection  process  that  was  used. 

•  State  the  proposal  evaluation  factors  and  subfactors. 

•  Reveal  the  evaluation  results: 

(1)  The  significant  advantages  of  the  awardee’s  proposal. 

(2)  The  significant  weaknesses  of  the  awardee’s  proposal. 

(3)  The  evaluation  ratings  of  the  awardee’s  proposal  to  the  second  level  of 
evaluation.  Explain  the  rating  definitions. 

(4)  The  Government’s  total  evaluated  costs/prices  of  the  awardee’s  proposal  for 
each  Contract  Line  Item  (CLIN).  Explain  significant  cost  realism  adjustments 
made  by  the  Government  to  the  major  cost  element  level. 
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A  summary  of  the  rationale  for  the  contract  award  decision. 

The  overall  ranking  of  all  proposals  (if  overall  rankings  were  made  during  source 
selection),  but  do  not  identify  the  unsuccessful  offerors  by  name. 

Answer  relevant  questions  pertaining  to  whether  the  Government  followed  the 
source  selection  procedures  set  forth  in  the  solicitation,  applicable  regulations,  and 
other  authorities. 


The  Postdebriefing  Memorandum 

The  contracting  officer  must  include  a  summary  of  each  debriefing  in  the  contract  file.  Good 
postdebriefing  memorandums  are  essential  if  the  acquisition  is  reopened  or  resolicited  as  a  result 
of  a  protest  or  otherwise  within  1  year  of  the  contract  award  date. 

In  those  circumstances,  the  law  requires  that  the  contracting  agency  make  available  to  all 
offerors  information  regarding  the  proposal  of  the  awardee  that  was  provided  to  other  offerors  at 
debriefings  on  the  prior  contract.  This  requirement  is  designed,  in  part,  to  place  all  offerors  on  a 
level  playing  field.  Thus,  the  need  for  good  postdebriefing  memorandums  is  apparent. 

The  postdebriefing  memorandum  should  include  at  a  minimum: 

•  A  list  of  all  persons  who  attended  the  debriefing. 

•  A  summary  of  the  information  disclosed  during  the  debriefing.  The  most  efficient 
means  for  doing  this  is  to  identify  the  charts  that  were  used  at  the  debriefing  and 
attach  a  copy  of  them  to  the  memorandum. 

•  The  substance  of  all  questions  and  answers  discussed  at  the  debriefing.  Include 
answers  provided  after  the  debriefing. 
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APPENDIX  C. 

PROTESTS  OF  CONTRACTS  AWARDED  USING  A  TRADE-OFF  PROCESS 


GAO  Case 


Number 

Case  Name 

Decision  Date 

Agency 

Decision 

283057 

Ocean  House  Builders 

21 -Sep-99 

Air  Force 

Denied 

278732 

Strategic  Resource  Solutions  Corp. 

09-Mar-98 

Air  Force 

Denied 

280758 

Avtron  Manufacturing,  Inc. 

16-Nov-98 

Air  Force 

Denied 

281200 

CDS  Network  Systems,  Inc. 

21 -Dec-98 

Air  Force 

Denied 

281046 

PCT  Services,  Inc. 

22-Dec-98 

Air  Force 

Denied 

281222 

WRS  Infrastructure  &  Environment,  Inc. 

12-Jan-99 

Air  Force 

Denied 

281275 

QSS  Information  Systems,  Inc. 

19-Jan-99 

Air  Force 

Denied 

281350 

Saratoga  Medical  Center,  Inc. 

27-Jan-99 

Air  Force 

Denied 

280521.3 

RMS  Information  Systems,  Inc. 

21 -Oct-98 

Air  Force 

Denied 

283011 

Engineered  Air  Systems,  Inc. 

21 -Sep-99 

AiT  Force 

Denied 

280521.2 

Techniacts  Science  &  Technology  Corp. 

15-Oct-98 

Air  Force 

Denied 

283028 

Premier  Engineering  &  Manufacturing 

Inc. 

21 -Sep-99 

Air  Force 

Denied 

283080 

Acepex  Management  Corp. 

04-Oct-99 

Air  Force 

Denied 

283272 

Alaskan  Publications 

27-Oct-99 

Air  Force 

Denied 

278591 

Beneco  Enterprises,  Inc. 

01 -Feb-98 

Air  Force 

Denied 

278582.2 

CPAD  Technologies,  Inc. 

19-Feb-98 

Air  Force 

Denied 

283794 

Tony’s  Landscaping,  Inc. 

23-Nov-99 

Air  Force 

Denied 

278695 

Modem  Technologies  Corp.,  et.al. 

04-Mar-98 

Air  Force 

Denied 

282272 

Inland  Service  Corp. 

21-Jun-99 

Air  Force 

Denied 

279839 

Rothe  Development,  Inc. 

27-Jul-98 

Air  Force 

Denied 

281359 

West  Coast  Research  Corp. 

01 -Feb-99 

Air  Force 

Denied 
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GAO  Case 
Number 

Case  Name 

Decision  Date 

Agency 

Decision 

281388 

OMV  Medical,  Inc.,  et.al. 

03-Feb-99 

Air  Force 

Denied 

281387 

OMV  Medical,  Inc.,  et.al. 

03-Feb-99 

Air  Force 

Denied 

281453 

Orbital  Technologies  Corp. 

17-Feb-99 

Air  Force 

Denied 

281965 

ENMAX  Corp. 

12 -May-99 

Air  Force 

Sustained 

280996.2 

NAPA  Supply  of  Grand  Forks,  Inc. 

13 -May-99 

Air  Force 

Denied 

282064 

Si-Nor,  Inc. 

25-May-99 

Air  Force 

Sustained 

280154.2 

Rohmann  Services,  Inc. 

16-Nov-98 

Air  Force 

Denied 

279173.5 

Acepex  Management  Corp. 

22-Jul-98 

Air  Force 

Denied 

282373 

Spectrum  Sciences  &  Software,  Inc. 

22-Jun-99 

Air  Force 

Denied 

279892 

Cache  Box,  Inc. 

29-Jul-98 

Air  Force 

Denied 

278855.2 

Apache  Enterprises,  Inc. 

30-M-98 

Air  Force 

Denied 

279191.3 

Main  Building  Maintenance,  Inc. 

05-Aug-98 

Air  Force 

Denied 

2784246.1 

Symetrics  Industries,  Inc. 

17 -Sep-9  8 

Air  Force 

Denied 

278478.4 

CVB  Company 

21 -Sep-98 

Air  Force 

Denied 

280374 

B3H  Corp. 

23-Sep-98 

Air  Force 

Denied 

280521 

Bionetics  Corp. 

14-Oct-98 

Air  Force 

Denied 

282082 

Buckeye  Park  Services,  Inc. 

Ol-Jun-99 

Air  Force 

Denied 

279251.2 

Exploration  Products 

01-Jun-98 

Air  Force 

Denied 

278897.4 

WP  Photographic  Services 

12-May-98 

Air  Force 

Denied 

278165 

International  Consultants,  Inc.,et.al. 

05-Jan-98 

Air  Force 

Denied 

279168 

PCT  Services,  Inc. 

12-May-98 

Air  Force 

Denied 

275587.9 

Pemco  Aeroplex,  Inc.,  Aero  Corp. 

29-Jun-98 

Air  Force 

Denied 

279811 

Applied  Companies 

24-Jul-98 

Army 

Denied 

280362 

Caterpillar,  Inc. 

23-Sep-98 

Army 

Denied 

278408 

Thermolten  Tech.,  Inc. 

26-Jan-98 

Army 

Denied 
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GAO  Case 
Number 

Case  Name 

Decision  Date 

Agency 

Decision 

278408.3 

Teledyne-Commodore,  LLC 

15-Sep-98 

Army 

Dismissed 

281136 

AMS  Mechanical  Systems,  Inc. 

04-Jan-99 

Army 

Denied 

278129.4 

Proteccion  Total/Magnum  Security,  S.A. 

12-May-98 

Army 

Denied 

278651 

Compro  Computer  Services,  Inc. 

23 -Feb-9  8 

Army 

Denied 

280133 

Electronic  Data  Systems  Corp. 

03-Sep-98 

Army 

Denied 

283307 

ITT  Federal  Services  International  Corp. 

03-Nov-99 

Army 

Sustained 

278358 

Compania  De  Asesoria  Y  Comercio,  S.A. 

20-Jan-98 

Army 

Denied 

283393 

Charleston  Marine  Containers,  Inc. 

08-Nov-99 

Army 

Denied 

280520 

Buck  Environmental  Technologies,  LLC 

14-Oct-98 

Army 

Denied 

278195.2 

Robotic  Systems  Technology 

07-Jan-98 

Army 

Denied 

278673 

Battelle  Memorial  Institute 

27-Feb-98 

Army 

Denied 

283259 

Govt,  of  Hartford  County,  Maryland 

2  8 -Oct-99 

Army 

Denied 

279561 

Trifax  Corp. 

29-Jun-98 

Army 

Sustained 

280422.3 

RTF/TCI/EAI  Joint  Venture 

29-Dec-98 

Army 

Denied 

282334 

Canadian  Commercial  Corp./Liftking 
Industries,  Inc. 

30-Jun-99 

Army 

Denied 

279723 

Riveer  Company 

14-JuI-98 

Army 

Denied 

282162 

TRW,  Inc. 

09-Jun-99 

Army 

Denied 

280978.3 

The  Arora  Group,  Inc. 

27-Jan-99 

Army 

Denied 

281342 

Lance  Ordnance,  Inc. 

26-Jan-99 

Army 

Denied 

282511 

ATA  Defense  Industries,  Inc. 

21-Jul-99 

Army 

Sustained 

278298 

Midmark  Corp. 

14-Jan-98 

Army 

Denied 

281246 

Vision  Blocks,  Inc. 

14-Jan-99 

Army 

Denied 

283401 

Olin  Corp. 

15-Nov-99 

Army 

Denied 

281006 

RMG  Systems,  Ltd. 

18-Dec-98 

Army 

Denied 
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GAO  Case 
Number 

Case  Name 

Decision  Date 

Agency 

Decision 

280988 

NWT,  Inc.,  PharmChem  Laboratories, 

Inc. 

17-Dec-98 

Army 

Denied 

280585 

Input/Output  Technology,  Inc. 

21 -Oct-98 

Army 

Denied 

280568 

Day  &  Zimmerman/IMR,  LLC 

19-Oct-98 

Army 

Denied 

279393 

Marlen  C.  Robb  &  Son  Boatyard  & 

Marina  Inc. 

09-Jun-98 

Army 

Denied 

278324.2 

Noelke  GmbH 

09-Feb-98 

Army 

Denied 

278319 

Electro-Voice,  Inc. 

15-Jan-98 

Army 

Denied 

281289 

APTUS  Company 

20- Jan-9  9 

Army 

Denied 

278168 

Joint  Threat  Services 

05-Jan-98 

Army 

Denied 

281331.3 

Kellie  W.  Tipton  Construction 

Company 

22-Mar-99 

Army 

Denied 

278797 

Rockhill  Industries,  Inc. 

16-Mar-98 

Army 

Denied 

278408.5 

Teledyne-Commodore,  L.L.C. 

08-Mar-99 

Army 

Denied 

278689 

Jason  Associates,  Corp. 

02-Mar-98 

Army 

Denied 

279561.2 

Professional  Performance  Development 
Group,  Inc. 

06-Jul-98 

Army 

Denied 

278730 

FC  Business  Systems,  Inc. 

06-Mar-98 

Army 

Sustained 

281141.3 

Clean  Service  Company,  Inc. 

16-Feb-99 

Army 

Denied 

277827.5 

Marquette  Medical  Systems,  Inc. 

29-Apr-99 

Army 

Sustained 

280766.3 

E.L.  Hamm  &  Associates,  Inc. 

12-Apr-99 

Army 

Sustained 

279637.2 

Hugo-Cantu  Joint  Venture 

20-Jul-98 

Army 

Denied 

283827 

B.  Diaz  Sanitation  Inc. 

27-Dec-99 

Army 

Denied 

277241.2 

Aalco  Forwarding,  Inc.,  etal. 

Ol-Jul-98 

Army 

Denied 

279585 

IT  Facilities  Services 

29-Jun-98 

Army 

Denied 

281384 

Bulora  Technologies,  L.L.C. 

03 -Feb-99 

Army 

Denied 

281002.2 

Andrulis  Corp. 

02-Jun-99 

Army 

Denied 

278793 

Coleman  Research  Corp. 

I6-Mar-98 

Army 

Denied 
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277241.15 

Aalco  Forwarding,  Inc.,  et.  al. 

11 -Mar-98 

Army 

Denied 

281244.3 

D  &  K  Construction  Company,  Inc. 

01 -Oct-99 

Army 

Denied 

282739 

Allied  Technology  Group,  Inc. 

19-Aug-99 

Army 

Denied 

278371 

Hardcore  Dupont  Composites,  L.L.C. 

20-Jan-98 

Army 

Denied 

282140 

North  State  Resources,  Inc. 

07-Jun-99 

Army 

Denied 

280044.2 

HAP  Construction,  Inc. 

21 -Sep-98 

Army 

Denied 

279492.2 

Black  &  Veatch  Special  Projects  Corp. 

26-Jun-98 

Army 

Denied 

282231 

Garco  Construction,  Inc.,  etal. 

1 5-Jun-99 

Army 

Denied 

281082 

Omni  Corp. 

22-Dec-98 

Army 

Dismissed 

281695 

Stratus  Systems,  Inc. 

24-Feb-99 

DoT 

Denied 

283882 

Phantom  Products,  Inc. 

30-Dec-99 

DoT 

Denied 

278422 

Baldt,  Inc. 

28-Jan-98 

DoT 

Denied 

279217 

Quaker  Valley  Meats,  Inc./Supreme  Sales, 
Upper  Lakes  Foods 

20-May-98 

DCA 

Denied 

280969 

MCR  Federal,  Inc. 

14-Dec-98 

DFAS 

Sustained  in  part 

280463.3 

Rel-Tek  Systems  &  Design,  Inc. 

25-Nov-98 

DFAS 

Sustained 

281393 

Deva  &  Associates,  PC. 

01 -Feb-99 

DFAS 

Denied 

283491 

Communications  Technologies,  Inc. 

30-Nov-99 

DISA 

Denied 

281876 

Marathon  Watch  Company  Ltd. 

22-Apr-99 

DLA 

Denied 

282554 

Wirt  Inflatable  Specialists,  Inc. 

28-Jul-99 

DLA 

Denied 

280180.2 

The  Terxe  Group,  Inc. 

28-Sep-98 

DLA 

Denied 

278268.2 

MCA  Research  Corp. 

10-Apr-98 

DLA 

Denied 

278881 

San  Diego  Beverage  &  Cup,  Inc. 

24-Mar-98 

DLA 

Denied 

280872.2 

Venture  Productions 

09-Dec-98 

DLA 

Denied 

282150 

Wellco  Enterprises,  Inc. 

04-Iun-99 

DLA 

Denied 

281958 

National  Aerospace  Group,  Inc. 

10-May-99 

DLA 

Sustained  in  part 
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280767.4 

D.F.  Zee’s  Fire  Fighter  Catering 

10-Sep-99 

DoA 

Denied 

280767.2 

Wilderness  Mountain  Catering 

28-Dec-98 

DoA 

Denied 

280218 

Sawtooth  Enterprises,  Inc. 

07-Dec-98 

DoA 

Denied 

279272 

William  C.  Menefee 

28-May-98 

DoA 

Denied 

281497 

Crasehi  Rentals,  Inc. 

17-Feb-99 

DoA 

Denied 

277063.3 

Tri-J  Contractors 

06-Jul-98 

DoA 

Denied 

281070.2 

West  Coast  Unlimited 

1 8-Aug-99 

DoA 

Denied 

283177 

GSA,  Inc. 

18-Oct-99 

DoA 

Denied 

278579.2 

Possehn  Consulting 

29-Jul-98 

DoA 

Sustained 

280774.2 

R.C.O.  Reforesting 

24-Nov-98 

DoA 

Denied 

278659 

Seedburo  Equipment  Co. 

25-Feb-98 

DoA 

Denied 

281706 

LB&B  Associates,  Inc. 

24-Mar-99 

DoA 

Denied 

277572.4 

HG  Properties  A.,  L.P. 

11 -Feb-98 

DoA 

Denied 

278278 

Biospherics,  Inc. 

14-Jan-98 

DoA 

Sustained 

278649 

High  County  Contracting 

30-Jan-98 

DoA 

Denied 

278579 

Possehn  Consulting 

09-Jan-98 

DoA 

Sustained 

283137.3 

Kathpal  Technologies,  Inc.,  etal. 

30-Dec-99 

DoC 

Sustained 

280003 

Madico,  Inc. 

12-Aug-98 

DoD  (HQ) 

Denied 

278904.4 

Centech  Group,  Inc. 

13-Apr-98 

DoE 

Denied 

278904.3 

Techsys  Corp. 

13-Apr-98 

DoE 

Denied 

278904.2 

McNeil  Technologies,  Inc. 

02-Apr-98 

DoE 

Denied 

277801.4 

Advanced  Data  Concepts,  Inc. 

Ol-Jun-98 

DoE 

Denied 

278044.5 

L  &  M  Technologies,  Inc. 

08-May-98 

DoE 

Denied 

282326 

Johnson  Controls,  Inc. 

28-Jun-99 

DoE 

Denied 
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280967.8 

Advanced  Data  Concepts,  Inc. 

14-Jun-99 

DoE 

Denied 

278451 

COMPA  Industries,  Inc. 

30-Jan-98 

DoE 

Denied 

278904 

KRACorp. 

02-Apr-98 

DoE 

Denied 

277805.2 

Scientech,  Inc. 

20-Jan-98 

DoE 

Sustained 

278869 

MicroMass,  Inc. 

24-Mar-98 

DoE 

Denied 

278044.4 

Source  One  Management,  Inc. 

12-Jun-98 

DoE 

Denied 

280929 

Westinghouse  Government  & 

Environmental  Services  Co,  Inc. 

04-Dec-98 

DoE/Navy  (joint) 

Denied 

280447 

YKK  (USA),  Inc. 

28-Aug-98 

DoJ 

Denied 

281074.2 

National  Toxicology  Laboratories,  Inc. 

11 -Jan-99 

DoJ 

Denied 

279048.2 

Professional  Gunsmithing,  Inc. 

24-Aug-98 

DoJ 

Denied 

282139 

Interlog,  Inc. 

27-Apr-99 

DoJ 

Denied 

280388 

General  Security  Services  Corp. 

25-Sep-98 

DoJ 

Denied 

281484.2 

Medical  Development  International 

29-Mar-99 

DoJ 

Denied 

280595 

Keane  Federal  Systems,  Inc. 

23-Oct-98 

DoJ 

Denied 

280959 

General  Security  Services,  Corp. 

11 -Dec-98 

DoJ 

Denied 

281051.5 

Warvel  Productions,  Inc. 

07-Jul-99 

DoJ 

Denied 

276633.2 

RAMCOR  Services  Group,  Inc. 

23-Mar-98 

DoJ 

Denied 

278906 

M3  Corp. 

01 -Apr-98 

DoT 

Denied 

278276.11 

Patriot  Contract  Services,  LLC,  et.al. 

22-Sep-98 

DoT 

Denied 

278466 

Hughes  STX  Corp. 

02-Feb-98 

DoT 

Sustained 

280270 

Randolph  Engineering  Sunglasses 

10-Aug-98 

DLA 

Denied 

280645 

Phillips  Industries,  Inc. 

l7-Sep-98 

DLA 

Denied 

281517 

Island  Components  Group,  Inc. 

19-Feb-99 

DLA 

Denied 

279697 

Dynamic  Marketing  Services,  Inc. 

13-Jul-98 

Education 

Denied 

209 


GAO  Case 


Number 

Case  Name 

Decision  Date  Agency 

Decision 

277029.3 

McAllister  &  Associates,  Inc. 

1 8-Feb-98 

Education 

Denied 

279924 

Information  Ventures,  Inc. 

07-Aug-98 

EPA 

Denied 

279379.2 

TEAM  Support  Services,  Inc. 

22-Jun-98 

EPA 

Denied 

280011.2 

Nick  Chorak  Mowing 

01  Oct-98 

EPA 

Denied 

278650 

USA  Today 

20-Feb-98 

DoJ 

Denied 

281836 

Construction  Technology  Laboratories, 

Inc. 

12-Apr-99 

DoT 

Denied 

283538 

Simborg  Development,  Inc. 

07-Dec-99 

GSA 

Denied 

279715 

McHargue  Construction  Co. 

16-Jul-98 

GSA 

Denied 

281995 

Concepts  Building  Systems,  Inc. 

13-May-99 

GSA 

Denied 

280518 

Acquest  Uniland  LLC 

13-Oct-98 

GSA 

Denied 

283398 

Ideal  Electronic  Security  Company,  Inc. 

10-Nov-99 

GSA 

Denied 

283192 

S.J.  Thomas  Co.,  Inc. 

20-Oct-99 

GSA 

Sustained 

277874.4 

Correa  Enterprises,  Inc. 

13-May-98 

GSA 

Denied 

276837.3 

Omega  World  Travel,  Inc. 

09-Jun-98 

GSA 

Denied 

279295 

Dual,  Inc. 

Ol-Jun-98 

GSA 

Denied 

278876.2 

NAAB  Research  Center,  Inc. 

04-May-98 

GSA 

Denied 

278851 

McShade  Enterprises 

23-Mar-98 

GSA 

Denied 

282776 

U.S.  Constructors,  Inc. 

21-Jul-99 

GSA 

Denied 

276012.2 

Wackenhut  Services,  Inc. 

01-Sep-98 

GSA 

Denied 

280475 

Cobra  Technologies,  Inc. 

06-Oct-98 

GSA 

Denied 

282456 

Universal  Building  Maintenance,  Inc. 

15-Jul-99 

GSA 

Sustained 

280013.2 

Information  Systems  Technology  Corp. 

06-Aug-98 

HHS 

Denied 

280176 

California  Resources 

08-Sep-98 

HHS 

Denied 

278508.4 

Biospherics,  Inc. 

06-Oct-98 

HHS 

Sustained 
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279260 

Conwal,  Inc. 

26-May-98 

HHS 

Denied 

282074 

GCI  Information  Services,  Inc. 

28-May-99 

HHS 

Denied 

278343 

Comark  Federal  Systems 

20-Jan-98 

HHS 

Sustained 

278678 

The  American  Indian  Center,  Inc. 

27-Feb-98 

HHS 

Denied 

278839 

I.S.  Grupe,  Inc. 

20-Mar-98 

HHS 

Denied 

279378 

Century  Environmental  Hygiene,  Inc. 

05-Jun-98 

HHS 

Sustained 

274689.5 

/ 

Tidewater  Homes  Realty,  Inc. 

ll-Aug-98 

HUD 

Denied 

279161.2 

Ervin  &  Associates 

1 3-Oct-98 

HUD 

Denied 

282098 

ACS  Government  Solutions  Group,  Inc. 

02-Jun-99 

HUD 

Sustained 

281352.14 

Housing  Services,  Inc. 

07-May-99 

HUD 

Denied 

278727 

U.S.  Property  Management  Service 

Corp. 

06-Mar-98 

HUD 

Sustained 

283147 

The  Communities  Group 

12-Oct-99 

HUD 

Denied 

279083 

Ervin  &  Associates,  Inc. 

30-Apr-98 

HUD 

Denied 

280993 

Ervin  &  Associates,  Inc. 

17-Dec-98 

HUD 

Denied 

280283.3 

Du  &  Associates,  Inc. 

22-Dec-98 

HUD 

Denied 

279565.2 

Consolidated  Engineering  Services,  Inc. 

26-Jun-98 

HUD 

Sustained 

279565.5 

Consolidated  Engineering  Services,  Inc. 

19-Mar-99 

HUD 

Denied 

282808 

Cotton  &  Company,  LLP 

30-Aug-99 

HUD 

Sustained 

281745.2 

Federal  Security  Systems,  Inc. 

29-Apr-99 

Interior 

Denied 

280478 

Emmert  International 

07-Oct-98 

Interior 

Denied 

282192 

Matrix  General,  Inc. 

IO-Jun-99 

Interior 

Denied 

283469.2 

Crescent  Helicopters 

30-Nov-99 

Interior 

Denied 

282661.2 

East  Slope  Conservation  Services 

09-Jul-99 

Interior 

Denied 

278450 

American  Artisan  Productions,  Inc. 

30-Jan-98 

Interior 

Denied 
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283822 

Century  Elevator  Inc. 

20-Dec-99 

JFK -CPA 

Denied 

277849.2 

Sytel,  Inc. 

08 -Jan-98 

NASA 

Denied 

278896.2 

Aerospace  Design  &  Fabrication 

04-May-98 

NASA 

Sustained 

283160 

Scientific  &  Commercial  Systems  Corp., 
et.al. 

14-Oct-99 

NASA 

Denied 

281773.2 

Mentor  Technologies,  Inc.,et.al. 

01 -Apr-99 

NASA 

Denied 

281773 

SGT,  Inc. 

01 -Apr-99 

NASA 

Denied 

280664.2 

Plasma-Therm,  Inc. 

28-Dec-98 

NASA 

Denied 

279828 

Technical  &  Administrative  Services 

Corp. 

24-Jul-98 

NASA 

Denied 

279014 

P  &  R  Water  Taxi,  Ltd. 

23-Apr-98 

Navy 

Dismissed 

278516.2 

Veda  Inc. 

19-Mar-98 

Navy 

Denied 

279919 

SDV  Telecommunications 

29-Jul-98 

Navy 

Denied 

281579 

Joppa  Maintenance  Company,  Inc. 

02 -Mar-99 

Navy 

Denied 

278260.2 

Davies  Rail  &  Mechanical  Works,  Inc. 

25-Feb-98 

Navy 

Denied 

279543 

Hard  Bodies,  Inc. 

23-Jun-98 

Navy 

Denied 

281490 

OMV  Medical,  Inc. 

16-Feb-99 

Navy 

Denied 

279759.2 

Systems  Integration  &  Research,  Inc., 
et.al. 

1 6-Feb-99 

Navy 

Denied 

280250 

Chant  Engineering  Co.,  Inc. 

07-Aug-98 

Navy 

Denied 

277538.2 

Command  Technology,  Inc. 

28-Apr-98 

Navy 

Denied 

279707 

HSQ  Technology 

09-Jul-98 

Navy 

Denied 

281521 

Chant  Engineering  Company,  Inc. 

22-Feb-99 

Navy 

Denied 

278187 

J  &  E  Associates,  Inc. 

05-Jan-98 

Navy 

Denied 

280048.4 

Pulau  Electronics  Corp. 

1 9-May-99 

Navy 

Sustained  in  part 

279493.3 

Telos  Corp. 

27-Jul-98 

Navy 

Denied 

283796 

B.E.  Meyers  &  Company,  Inc. 

14-Dec-99 

Navy 

Denied 
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278738 

Xeno  Technix,  Inc. 

11 -Mar-98 

Navy 

Denied 

279793 

Pacific  Ship  Repair  &  Fabrication,  Inc. 

23-Jul-98 

Navy 

Denied 

281872 

Metro  Machine  Corp. 

22-Apr-99 

Navy 

Sustained 

281784.3 

L-3  Communications  Corp,  Ocean 

Systems  Division 

26-Apr-99 

Navy 

Sustained  in  part 

282437.3 

Bums  &  Roe  Services,  Corp. 

30-Nov-99 

Navy 

Dismissed 

279773 

Southwest  Critical  Care  Associates 

16-Jul-98 

Navy 

Denied 

283650 

Advanced  Communication  Systems,  Inc. 

16-Dec-99 

Navy 

Denied 

279662.5 

Electronic  Design,  Inc. 

25-May-99 

Navy 

Denied 

283738 

BioGenesis  Pacific,  Inc. 

14-Dec-99 

Navy 

Denied 

283448 

J  &  E  Associates,  Inc. 

03-Nov-99 

Navy 

Denied 

280814 

Packaging  Strategies,  Inc. 

25-Nov-98 

Navy 

Denied 

280511.2 

GTS  Duratek,  Inc. 

19-Oct-98 

Navy 

Sustained 

282982 

Thames  Towboat  Company,  Inc. 

09-Sep-99 

Navy 

Denied 

281278 

Ventura  Petroleum  Services,  Inc. 

21 -Jan-99 

Navy 

Denied 

278141.2 

Dawco  Construction,  Inc. 

02-Jan-98 

Navy 

Denied 

281285 

Browning  Ferris  Industries  of  Hawaii, 

Inc. 

21 -Jan-99 

Navy 

Denied 

280853 

Durocher  Dock  &  Dredge/Black  & 

Veatch,  A  Joint  Venture 

24-Nov-98 

Navy 

Denied 

283236 

ELS,  Inc. 

25-Oct-99 

Navy 

Denied 

280573.2 

Environmental  Tectonics  Corp. 

01 -Dec-98 

Navy 

Denied 

280834 

Lear  Siegler  Services,  Inc. 

25-Nov-98 

Navy 

Denied 

278281 

F2M-WSCI 

14-Jan-98 

Navy 

Denied 

282670 

A  &  T  Engineering  Technologies 

13-Aug-99 

Navy 

Denied 

282271 

Am  Clyde  Engineered  Products 

Company,  Inc. 

21-Jun-99 

Navy 

Denied 

277143.2 

Navistar  Marine  Instrument  Corp. 

13-Feb-98 

Navy 

Denied 
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282887 

Olympus  Building  Services,  Inc. 

31 -Aug-99 

Navy 

Denied 

282912 

Infrared  Technologies  Corp. 

02-Sep-99 

Navy 

Denied 

283239 

DTH  Management  JV 

06-Oct-99 

Navy 

Denied 

282630.2 

OMNIPLEX  World  Services  Corp. 

22-Sep-99 

Navy 

Denied 

282947 

Nova  Group,  Inc. 

15-Sep-99 

Navy 

Sustained 

283234 

J.A.  Jones  Grupo  de  Serviciois,  SA. 

25-Oct-99 

Navy 

Denied 

277926.2 

Teleport  Communications  Group 

17-Sep-98 

Navy 

Denied 

280049 

Nations,  Inc. 

24-Aug-98 

Navy 

Sustained 

278584 

U.S.  Technology  Corp. 

17-Feb-98 

Navy 

Denied 

279662.2 

Electonic  Design,  Inc. 

31 -Aug-98 

Navy 

Sustained 

278734.4 

East/West  Industries 

28-May-98 

Navy 

Denied 

281142 

National  Steel  &  Shipbuilding  Co. 

04-Jan-99 

Navy 

Denied 

280261 

Morrison  Knudsen  Corp. 

09-Sep-98 

Navy 

Denied 

281437.2 

Wilcox  Industries,  Corp. 

30-Jun-99 

Navy 

Sustained 

280284 

Aerostructures,  Inc. 

15-Sep-98 

Navy 

Denied 

278647.2 

Oceanometrics,  Inc. 

09-Jun-98 

Navy 

Denied 

282249 

J  &  D  Maintenance  &  Service 

18-Jun-99 

Navy 

Denied 

282407 

Support  Services,  Inc. 

08-Jul-99 

Navy 

Denied 

278921.2 

Braswell  Services  Group,  Inc. 

17-Jun-98 

Navy 

Denied 

282634 

Red  River  Service  Corp. 

15-Jul-99 

Navy 

Denied 

278360 

Tidewater  Construction  Corp. 

20-Jan-98 

Navy 

Sustained 

280398.2 

Knights'  Piping  Inc.,et.al. 

09-Oct-98 

Navy 

Denied 

281030 

Beckman  Coulter 

21 -Dec-98 

HHS 

Sustained 

279049 

Chant  Engineering  Company 

30-Apr-98 

PCC 

Denied 

214 


GAO  Case 


Number 

Case  Name 

Decision  Date  Agency 

Decision 

277916.2 

Mechanical  Contractors,  S.A. 

04-Mar-98 

PCC 

Sustained 

281503 

The  Production  Company 

18-Feb-99 

SS  Admin. 

Denied 

279305 

WECO  Cleaning  Specialists,  Inc. 

03-Jun-98 

SS  Admin. 

Denied 

280429 

Securiguard/Group  4  Joint  Venture 

30-Sep-98 

State 

Denied 

278929.7 

Jensco  Marine,  Inc. 

11 -Feb-99 

Treasury 

Denied 

275502.3 

Signal  Corp. 

06-Jul-98 

Treasury 

Denied 

281255 

Amdahl  Corp. 

28-Dec-98 

Treasury 

Denied 

279221 

Ideal  Electronic  Security 

19-May-98 

Treasury 

Denied 

280922 

Teltara,  Inc. 

04-Dec-98 

Treasury 

Sustained 

278929.2 

Mar,  Inc.,etal. 

28-Sep-98 

Treasury 

Denied 

281919 

KBM  Group,  Inc. 

03 -May-99 

Treasury 

Denied 

278715 

The  Centech  Group,  Inc. 

05-Mar-98 

Treasury 

Denied 

278189.3 

Foundation  Health  Federal  Services,  Inc. 
et.al. 

04-Feb-98 

TSO 

Sustained 

278254 

Research  Triangle  Institute 

12-Jan-98 

USAID 

Denied 

279920 

Development  Alternatives,  Inc. 

06-Aug-98 

USAID 

Denied 

281241.2 

Ritchie  Sawyer  Corp. 

25-Jan-99 

USAID 

Denied 

281028.2 

Mid-Atlantic  Service  &  Supply  Corp. 

29-Dec-98 

USAID 

Denied 

280113 

RONCO  Consulting  Corp. 

11 -Aug-98 

USAID 

Denied 

282555 

Chemonics  International,  Inc. 

23-Jul-99 

USAID 

Sustained 

282571 

AdvanChip  Corp. 

29-Jul-99 

Interior 

Denied 

279756 

Robertson  Leasing  Corp. 

17-Jul-98 

DoJ 

Denied 

280959.5 

Famham  Security,  Inc. 

09-Feb-99 

DoJ 

Denied 

281904 

Walsh  Distribution,  Inc. 

29-Apr-99 

VA 

Denied 
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280430 

SDS  Petroleum  Products,  Inc. 

2822243 

Life  Oxygen  &  Health  Services,  Inc. 

281693 

Op ti -Lite  Optical 

278223 

BFI  Waste  Systems  of  Nebraska,  Inc. 

278400 

University  of  Kansas  Medical  Center 

281693.2 

Opti-Lite  Optical 

281800 

Korrect  Optical 

280652 

HG  Properties  A.,  L.P. 

283295.2 

Rotech  Medical  Corp. 

Decision  Date  Agency  Decision 


01 -Sep-98 

VA 

Denied 

18-Jun-99 

VA 

Denied 

22-Mar-99 

VA 

Sustained 

08-Jan-98 

VA 

Denied 

26-Jan-98 

VA 

Denied 

15-Jul-99 

VA 

Denied 

09-Apr-99 

VA 

Denied 

02-Nov-98 

VA 

Denied 

08-Nov-99 

VA 

Denied 
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